Re: [Roll] IOTdir Review of draft-ietf-roll-dao-projection-19

"Pascal Thubert (pthubert)" <pthubert@cisco.com> Tue, 21 September 2021 16:13 UTC

Return-Path: <pthubert@cisco.com>
X-Original-To: roll@ietfa.amsl.com
Delivered-To: roll@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC2173A0035; Tue, 21 Sep 2021 09:13:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.599
X-Spam-Level:
X-Spam-Status: No, score=-9.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=jQ7FwT/f; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=D4E972e+
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QZl9cR2bDFYB; Tue, 21 Sep 2021 09:13:13 -0700 (PDT)
Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B0583A1E9D; Tue, 21 Sep 2021 09:13:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=365508; q=dns/txt; s=iport; t=1632240793; x=1633450393; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=WLgeAujks43YxDoXAuTf3fLk8jdqw1JT4YG0yo3XxLk=; b=jQ7FwT/fmzl2yxe2/zWPoHBcvAkqYtTT1z+2u8euG8i//qBR5fSbkZYn m6QO0QEQzWmm8QlaE8JnQ49ej3JYbbrEy78VDGabYrwFmABOjnk0h2hmD usGbVWjLr8WcAjR244S4Zb1/wvahjVuNEsmkCwjkdwheXZvjYpz5L2l8l Q=;
X-IPAS-Result: A0DmAgBJBEphl4cNJK2FYLZ6h0cnTAYBAQEBAwIDAQEBAQwDAgEGBBUBAQICgQmMfjqEQbRjiW6SGIHoWRkP
IronPort-PHdr: A9a23:5QSHRxTBQb8wu4FnLb3vJl/um9pso6HLVj580XJvo7lVNKqq4tLuM R+X6fZsiQrPWoPWo7JBhvHNuq/tEWoH/d6asX8EfZANMn1NicgfkwE6RsLQD0r9Ia3hdGo0F dkEWFI2t32+OFJeTcD5YVCaq3au7DkUTxP4Mwc9Jun8FoPIycqt0OXn8JzIaAIOjz24MttP
IronPort-Data: A9a23:W66R2K/3CL8cZGCPaa1wDrUDkH6TJUtcMsCJ2f8bNWPcYEJGY0x3x mYZWTzQOvaIazCgcoxyPIy0pEoCvJ/SytJhHlRr+yhEQiMRo6IpJzg2wmQcns+2BpeeJK6yx 5xGMrEsFC23J5Pljk/F3oLJ9RGQ7onVAOqhYAL4EnopH1Y8GX140UgLd9MR2+aEv/DoW2thh vuqyyHvEAfNN+lcaz98Bwqr8XuDjdyq0N8qlgVWicNj4Dcyo0Io4Kc3fsldGZdXrr58RYZWT 86bpF2wE/iwEx0FUrtJmZ6jGqEGryK70QWm0hJrt6aebhdq/gls1LoANesgNk59qjiExdky0 PhVusnlIespFvWkdOU1Wh1cFWR1OrdLveWBKnmkusvVxErDG5fu66wxVwdtY8tBoaAuWjwmG f8wcFjhajibm+Kryr+hVsFnh98oK4/gO4Z3VnRIkm6GUKl7G8iTK0nMzc1W/xYPwZkVJ+v1R MZCNjl+SjLZeBIabz/7D7pnzLv32RETaQZwpEicq7Zy4mXPwklr17zpddbcfZmSX8JXk02Ep 2XAuW39BjkbOcCRjz2f/RqEnOjLmQv6VZ4cUrqi+ZZCiUee2mUVAVsXWEe1ifa8g0+6HdlYL iQ85jYjhaMpskKiU5/xUnWFTGWstxoYXZ9bFPc3rVvLwavP6AHfDW8BJtJcVDA4nJcTGzo27 V6Yo4zgLx1iveecCmqw7rjB+FteJhMpwX8+iT4sFFVeuYCz/t1r03ojXf44S/bk1I2d9SXYh mHU8nRj293/mOZRj82GEUb7byVAT3QjZicx4gjRNo5OxlwkPNb/D2BEBKSy0BqtBI+dSl/Et 38elo3EhAzvMX1vvHHWKAnuNOj0jxpgDNE7qQU1d6TNDxz3pxaekXl4uVmS3nuF1/ronxe0P ic/XisPvPdu0IeCN8ebnqroUZ1xlPi8fTgbfq+EN7KinaSdhCferH0xOiZ8LkjGkVMnlukEK IyHfMO3ZUv2+ow2lWbmGb917FPf/QhnnTm7bcmil3yPiOPCDFbIGeZtGAbfNYgRsfLbyC2Lq Iw3H5XRlH1ivBjWP3C/HXg7dgtRcxDWxPne9qRqSwJ0ClA3QD1+U6CBn+xJlk4Mt/09q9okN 0qVAidwoGcTT1WZdG1mtlgLhGvTYKtC
IronPort-HdrOrdr: A9a23:wCNHTaEp30Nob5kbpLqFt5LXdLJyesId70hD6qkvc31om52j+f xGws516fatskduZJkh8erwX5VoMkmshKKdhrNhfotKPTOW+FdASbsD0WKM+UyaJ8VxnNQtr5 uIH5IObeEYSGIK8voSgzPIUerIouP3jZxA7N22pxwGIG0aCNAD0+46MHfmLqQcfnghOXNNLu vl2iMxnUvYRZ14VLXeOlA1G8z44/HbnpPvZhALQzQ97hOVsD+u4LnmVzCFwxY3SVp0sPIf2F mAtza8yrSosvm9xBOZ/XTU9Y5qlNzozcYGLNCQi/ISNi7nhm+TFcBcsvy5zXcISdOUmQ8Xee r30k8d1gNImijsl1SO0F3QMs/boWwTAjHZuAKlaDDY0LzErXoBerl8bMRiA0fkA45KhqAj7E qNtFjp6Ka/RCmw7hjV9pzGUQpnmVGzpmdnmekPj2ZHWY9bc7NJq5cDlXklXKvoMRiKorzPKt MeQf00JcwmB2+yfjTcpC1i0dasVnM8ElOPRVUDoNWc13xTkGpix0UVycQDljNYnahNBaVs9q DBKOBlhbtORsgZYeZ0A/oAW9K+DijITQjXOGyfLFz7HOUMOm7LqZTw/LIpjdvaN6Ag3d83gt DMQVlYvWk9dwbnDtCPxoRC9lTXTGC0TV3Wu4pjDlhCy/XBrZ/QQGy+oXwV4r+dSsQkc4Tmsq yISedr6tfYXBzTJbo=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos;i="5.85,311,1624320000"; d="scan'208";a="777575208"
Received: from alln-core-2.cisco.com ([173.36.13.135]) by alln-iport-5.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 21 Sep 2021 16:13:07 +0000
Received: from mail.cisco.com (xbe-rcd-002.cisco.com [173.37.102.17]) by alln-core-2.cisco.com (8.15.2/8.15.2) with ESMTPS id 18LGD5v0008798 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=OK); Tue, 21 Sep 2021 16:13:07 GMT
Received: from xfe-rtp-001.cisco.com (64.101.210.231) by xbe-rcd-002.cisco.com (173.37.102.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15; Tue, 21 Sep 2021 11:13:06 -0500
Received: from xfe-aln-002.cisco.com (173.37.135.122) by xfe-rtp-001.cisco.com (64.101.210.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15; Tue, 21 Sep 2021 12:13:05 -0400
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (173.37.151.57) by xfe-aln-002.cisco.com (173.37.135.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15 via Frontend Transport; Tue, 21 Sep 2021 11:13:05 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kd/q4gAwicBFzMQ/T0/zpU6re3ilOCX46vgjXYnQQDddjOwGqKqOC7MldMpXpkNaUizHQnQZVjVOtndD3Za0SKa64ct/sbHSSutws7JzTEjNHLPDoiyQO8QMXuN18jL8VXUckQZ4kcXWxf17cvkbUNXhRMrwPiJc/fwd6WjN/qlOT55oLq4HPiRUXGj5dmyyQtuupzUbGAOvhGFjzUBk/Wd3rpfC1KtZFhXRlmfW25jv4KYrgxHWTP1Kw3/PlVxl/w80jgEn/x8CBrZ3oqZDePiU3vVkVSJ58M811vi4XwqWoTeuK8vVjnAmurITP/fKMJ3GhlixvdxWisILwrKWuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=WLgeAujks43YxDoXAuTf3fLk8jdqw1JT4YG0yo3XxLk=; b=n4C40b2UIyW5ujm5Cvu+MuW4PJym56Wva2po29h57SUW5YVEQXH/EBlqwh3vlVRplQMdpp1SIG3p9ZpBAajr+GxyhulQcvzNENnPhbgpzbTe3NnyGvTQ3/CGBJMjkYLZpF0aCymygInDMLxUG/24CWSoYIsaaidDpdIZiVb83TDsh/uXsIbbR1/J0pKxV5qtvcLtgwTyDeGFsA35uGd0kFI9HMvmOfVj6U69BkUqd6zSdp8X6eesPTYgwKyodf4kmsJilvSPS6TKtfyLwyivrt2o8uO1v/VyyxM+lqWLwG2iJOM3+BdGx75Xu+MRxUykJ75CVdV0VABO7ojyDlEfdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WLgeAujks43YxDoXAuTf3fLk8jdqw1JT4YG0yo3XxLk=; b=D4E972e+fK7VpnrDNw7nCHSFHacvz9m2SweOfKshxj1dRygaQlTlyjsPHcFFD3MlblgXgxi65t+NnA44aANY9dZAxi2Q/9RJ4sCnopVjBX3cfr1ENtJ+SWEYGpF8/XEgSComi7tDUzxY5Ng8+dHDlpBAZTdXCXnyE5u3zBQ5PHE=
Received: from SJ0PR11MB4896.namprd11.prod.outlook.com (2603:10b6:a03:2dd::20) by BYAPR11MB3304.namprd11.prod.outlook.com (2603:10b6:a03:7a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 21 Sep 2021 16:12:54 +0000
Received: from SJ0PR11MB4896.namprd11.prod.outlook.com ([fe80::8953:241d:7255:35be]) by SJ0PR11MB4896.namprd11.prod.outlook.com ([fe80::8953:241d:7255:35be%3]) with mapi id 15.20.4523.019; Tue, 21 Sep 2021 16:12:54 +0000
From: "Pascal Thubert (pthubert)" <pthubert@cisco.com>
To: Toerless Eckert <tte@cs.fau.de>, Alvaro Retana <aretana.ietf@gmail.com>
CC: Routing Over Low power and Lossy networks <roll@ietf.org>, "draft-ietf-roll-dao-projection.all@ietf.org" <draft-ietf-roll-dao-projection.all@ietf.org>
Thread-Topic: [Roll] IOTdir Review of draft-ietf-roll-dao-projection-19
Thread-Index: AQHXnSUJ2blEs70G1kOM/vt917Gg/6uL7K8Q
Date: Tue, 21 Sep 2021 16:12:25 +0000
Deferred-Delivery: Tue, 21 Sep 2021 16:12:12 +0000
Message-ID: <SJ0PR11MB489607966AAB35A77EAE7CE2D8A19@SJ0PR11MB4896.namprd11.prod.outlook.com>
References: <20210829222620.GA6858@faui48f.informatik.uni-erlangen.de>
In-Reply-To: <20210829222620.GA6858@faui48f.informatik.uni-erlangen.de>
Accept-Language: fr-FR, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: cs.fau.de; dkim=none (message not signed) header.d=none;cs.fau.de; dmarc=none action=none header.from=cisco.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b2fc4b2b-510d-46d2-730a-08d97d1aab47
x-ms-traffictypediagnostic: BYAPR11MB3304:
x-microsoft-antispam-prvs: <BYAPR11MB330450C95F0BABF1936147FBD8A19@BYAPR11MB3304.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: SUOzfsIWvEgE4Hikuit460wJqjRk+dFOq+T/wqwUC2vTFJJn4SG9r9o79UkfyXwe5HHNkaM/oVAGmwi5WM0kxul2na0FOJRPfl8fJhOFU5G3gss47js93UiLf5DcJzeHbqRDdgeTb8alMZs2dbA365xPzYgDMuMmyHxxNed8wpwm9uUREhqwJXOOTiKTNlEL3oSe0LzJ38YkgD07+3IdeAP04MSIQYYqZPGe7NqprO908lJrEsKhmos+zBQTrE+Qh7KmHlsHTY5ktgX9eCx+AZioKScJU1fq6OdsQy8mB9i5zvWAal3gGFxHlVkFNmtUeyn69Bf1vUYdMHKxxQPvBARhNT7lj9kv/vLEGToL9quSaxZZXHakxNpinmr2oiP68YXsgf3ED4cp0WNuc1ZfSvsPJmK5MCvtmd9eypCrXiWuACJymwWDY8sXnEJM71oYS0UmnE0xnj9KEjB+Tq0Y6RWXye9hmMmbxOXFTkIZblWZrc543n8VCV9vU7D0Rez9g6EZuZVSo4D5xrKbexCsn7SUAO/EvkiOSpvtEty0kcTBwiKc/zHkm5YZA62XbUeTtkDmcjGiQJjiI+E6JtGCdnvrqfnESNZJyQXyY1e5Qbw1hfd3WMw0KmndD5oQm7pRviI021BUZYEJf79pXaNUCENcWFaBUfj//i94grCYa3qw6kuMBwHz4dqZlAgg7XlkdA9A68jD5K+5MWVRim41FvZ6yHJQlQAlQqe7OvYiEP+B1nZg7l2lzbsvWDYGEeIVL9i2bwK+31dy2Ze54sC11gqp2EPi1JQYatmWd9U12bqSGhBKQE5LcLLxpHWEhhoCgXo2aP9AA1rcWyAd/pjfzQ==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB4896.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(316002)(122000001)(38100700002)(966005)(86362001)(2906002)(76116006)(186003)(8936002)(18074004)(6506007)(110136005)(9686003)(33656002)(5660300002)(8676002)(55016002)(52536014)(66446008)(64756008)(7696005)(66946007)(83380400001)(71200400001)(6666004)(4326008)(508600001)(38070700005)(40140700001)(66476007)(66556008)(54906003)(30864003)(66574015)(579004)(559001)(569008); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: Z5R6c3DSAQVzpHi0+k/hXi1millrH03z1F62LZXVkfBOIKOTjY+JDX/Yhr6PD9axpcKkKUucVVaSxthXJ6LLwwjuBfOgyjfQJ2e8VGPYt/dA9JlGFn3SlcMzvuPcANyZHRuYL0Bdvdivk/+Wajnzs+x5pVaCZYlIiN9XACGQxtpAUmbknICU4kI9MCswbDZH+a6moqemktZT4XZmOU1IGVfSLQfsnNCEQWyzSQhqYwYG0Qc5A89FadEJITG8ZFc6aFisotVlChR3tbi1bHpKaBZp5ILDyDydrqDXkxsFgVYhtVgjyiwRFODFwovQb0bM3E4uVKYccc54R3Usec2oLE/iSACL2p2xoYY1EPdxK0Ik54ukToZMPqv3jagv6FcIQa7uUq9ccSov77rEpI9Jd+GMDNu/nOCEmgqotfxZnYyDIzDZaBboKFA9ioqrUSKNDtdRgJ+djJeBdzVVejKfdFuoQZz9A9rnK+Zm50Wrev6YpcOeWIjUFOkuVMF+1DmxprfXR6xPNSFFiUoxwMO2lwSUX6XIc4e9B/PvCNQDTN2gJ4QYd3DeARxGfivABwTCRAWCCHwoeoKm4fkT1fbTQ2V5vjwktvwcOfhb0yKvbNy9Va+ZzR1+7vzPbgvxh6kv8v6VVwls9zJRv5TjL8KjGhRfP4D55CSXSNkfSoFS69NRa7ScaRiNwQSH7QZ8iJIysIdYQsnEal+1DoBpukbPzAOmabDJqj0cdpu57qlKtITdY3EGCrsk2A9d+DOVG4XpPNh7Uk+HdhcyMvApsYm9cRKGLl3EgPgttvGVJO5WhP2QcgEpJ8xNEB0aDBmQtbxVy4UzAAdGHVlXV10Q77AmP5BGxXhgI/wsNVNcSWEdVrZxFxW7+OtFhSCo6Qw0weAn8+95MNJWP9c07BTopV4V2MTDXdyhRmUX6h0rCKpwCkFSVclLeV/wRFvD9dr1sQ6y/On4WOz+FQIyT2jOUy45MjFSaYMAPEwYpiDQ6SeaiaxCfQnepdlsvIseTkuVK4YVEpPLSa8Lt43BLtqCEbruWjepTHA5J1Tj7PA+GlSzU+VQ51oMKdySkzyKCO+gxxLnQX9bTW+brfyQvgUAGy5jLdk5kIY83d103C7gqgv2d6RRgNREM+/R9N6/wggqor8Tx6KUK+I9AwkGGfAj9AyUgBY7UE5YPMkNFwyyZTczqmuVy4IgerCojoN7RzxU33RpiSMrd4LcAwf+2TYbinxpKJJpGKYl9WI+7mYtgD82IeNFv0khcLHfmhHQWuMTqK4NSMBlz2OXKJUOjB+McWvurreWIf2v53Z0Xi0uFuetBrU3M0gk4YeHzhdOFm2+W2Pn8o7hLPTIlRqqIFJ5JVVvSTQytywsr42xgyJnvXXNgc9COXqAV1CfSlc57k76JldF
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB4896.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2fc4b2b-510d-46d2-730a-08d97d1aab47
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 16:12:54.2070 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: srfkUIaeL/0hLwB2jctJyx2Kg9H4uyUd45aykaoKrTTIibnQ1SYxlSrRg5RedLDoRRt/GRmNXK9cSZTIdDO5ow==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3304
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.37.102.17, xbe-rcd-002.cisco.com
X-Outbound-Node: alln-core-2.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/roll/FcGB6cnL7gPvY2jk5LQF8s_5QE0>
X-Mailman-Approved-At: Wed, 22 Sep 2021 01:24:44 -0700
Subject: Re: [Roll] IOTdir Review of draft-ietf-roll-dao-projection-19
X-BeenThere: roll@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Routing Over Low power and Lossy networks <roll.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/roll>, <mailto:roll-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/roll/>
List-Post: <mailto:roll@ietf.org>
List-Help: <mailto:roll-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/roll>, <mailto:roll-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 21 Sep 2021 16:13:29 -0000

Hello Toerless:

Many thanks for your time and care in this review. It was huge, involved reorganizing the doc and I published 20 to settle a base line:
https://www.ietf.org/archive/id/draft-ietf-roll-dao-projection-20.html

Alvaro, can you please have a look at Toerless' argument about update vs. extend around line 486?

Let's see:
 
> Thanks a lot for this work.  This is cool technology, it has the
> ability to provide very flexible and wide range of path steering
> and header optimizations for RPL networks.
> 

Sure! A note that your review comments suggest important reorgs (not redesign!) in the doc, which are mostly applied as suggested.
This will make diff review very hard going from 19 to 20. I hope things get stable after that. The diff is here:
https://www.ietf.org/rfcdiff?url2=draft-ietf-roll-dao-projection-20

OTOH, As the review progressed, I had to change and again, so proposed text below may have been updated or moved afterwards.
So I guess the best will be for you to skim through 20 to assess the changes. On the bright side, I added text to support ACP direct paths using P-DAO. 

Win-win 😉


> Overview:
> 
> I am not sure i understand all finer details due likely
> to some impedance mismatch between the expected RPL expertise of the text
> and mine, but i think that the proposed technology is sound
> and has on the technoloy (not text) side only few details that may
>  need to be added (missing guidance or references on timers,
> maybe MTU handling if that is not supposed to be well-known, etc.).

All good points. For timers, RPL generally avoids absolute guidance though relative values of times may be indicated if there's a dependency. Because use cases vary so much.
For the MTU, I agree, let me add

                     In order to avoid fragmentation, e.g., using
   [RFC4944], [RFC8930], and/or [RFC8931] in a 6LoWPAN LLN, it is
   RECOMMENDED to allow an MTU that is larger than 1280 in the main
   DODAG and allows for the additional headers while exposing only 1280
   to the 6LoWPAN Nodes as indicated by section 4 of [RFC4944].


> 
> If there is any technological gap, then it is me hoping that this
> could be a solution for the RFC8994 A.9.4 problem, but alas this
> solution does not support establishment of additional routes for
> storage mode DODAGs. Too bad. Especially because its not quite clear
> to me why that would not be possible to easily do as well. But
> in understand how that RPL profile may not have been of any interest
> for this work.
> 

The only reason storing mode is excluded in this spec is simplification.
Because in that mode the Root is missing topology information. 
But that could be alleviated by indicating the parents as siblings.
I made a separate proposal based on your thread with Michael on ANIMA's need.
The proposal is a new section 10 that explains why the spec is designed for non storing in LLNs and how to use it in storing-mode non-LLNs.


> As i am not an expert for RPL, but mostly familia with only
> basic use of RPL, i am not sure how much of an expert a reader of this
> document is expected to be to be able to read, understand and vet this
> text.
> I think i and other RPL beginners would be able to understand the text
> and vet potential gotches better if more explanations where given,
> and my feedback indicates this accordingly. But i will not claim
> whether such explanations are always deemed necessary for the target
> audience.

Just what we need!

> 
> The mayor issue in the text is that it front-loads a lot of details
> that are then hard to understand and open questions are only answered
> much later. For example, packet encoding is put first in the document,
> as opposed to (as i have observed it in more RFCs) at the end. Then
> the text follows with processing description, and only finally with
> what i consider to be a complete description of how the pieces
> interact through examples. As mentioned in the details, i think the order
> should
> be as much as possible top-down, starting with sufficienly complex
> examples that introduce the technology to the extend that the
> main concepts are used in the example. E.g.: sequential or complex track,
> sements, destinations, targets.

OK, makes sense. I suggest to improve the terminology section to provide details on the main concepts as below:


2.3.  Main Concepts

   RPL is primarily designed to minimize the control plane activity,
   that is the relative amount of routing protocol exchanges vs. data
   traffic; this approach is beneficial for situations where the power
   and bandwidth are scarce (e.g., an IoT LLN where RPL is typically
   used today), but also in situations of high relative mobility between
   the nodes in the network (aka swarming, e.g., within a variable set
   of vehicles with a similar global motion, or a toon of drones), and
   in situations where the traffic is mostly directed from or to a
   central node, such as the control traffic between routers and a
   controller of a Software Defined Networking (SDN) infrastructure.

   To reduce the routing exchanges, RPL leverages an anisotropic
   Distance Vector approach, which does not need a global knowledge of
   the topology, and only optimizes the routes to and from the root,
   allowing P2P paths to be stretched.  Although RPL installs its routes
   proactively, it only maintains them lazily, in reaction to actual
   traffic, or as a slow background activity.  Additionally, RPL
   leverages the concept of objective function which allows to adapt the
   activity of the routing protocol to the use case, e.g., type, speed,
   and quality of the LLN links.  RPL does not need converge, and
   provides connectivity to most nodes most of the time.  The default
   route towards the Root is maintained aggressively and may change
   while a packet progresses without causing loops, so the packet will
   still reach the root.

   RPL first forms a default route in each node towards the a Root, and
   those routes together coalesce as a Directed Acyclic Graph upwards.
   RPL then constructs routes to so-called Targets in the reverse
   direction, down the same DODAG.

   In Non-Storing Mode, each node advertises itself as a target directly
   to the Root, indicating the parents that may be used to reach self.
   Recursively, the Root builds and maintains an image of the whole
   DODAG in memory, and leverages that abstraction to compute source
   route paths for the packets to their destinations down the DODAG.
   When a node changes its point(s) of attachment to the DODAG, it takes
   single unicast packet to the Root along the default route to update
   it, and the connectivity is restored immediately; this mode is
   preferable for use cases where internet connectivity is dominant, or
   when, like here, the Root controls the network activity in the nodes.

   In Storing Mode, the routing information percolates upwards, and each
   node maintains the routes to the subDAG of its descendants down the
   DODAG.  The maintenance is lazy, either reactive upon traffic or as a
   slow background process.  Packets flow via the common parent and the
   routing stretch is reduced vs. Non-Storing, for a better P2P
   connectivity, while the internet connectivity is restored more
   slowly, time for the DV operation to operate hop-by-hop.

   Either way, the stretched routing is not adapted to use cases that
   require high availability and reliability, such as; e.g., listed in
   RAW use cases [USE-CASES].  Extra stretch is counter-productive to
   both reliability and latency, and there's a need to better control
   the use of the available diversity, be it in time, frequency, path or
   technology.


   To complement RPL and eliminate routing stretch, this specification
   introduces an hybrid mode for RPL, that combines both Storing and
   Non-Storing modes to build Tracks from Ingress to Egress inside the
   DODAG.  A Track is typically a collection of parallel loose source
   routed sequences of nodes from Ingress to Egress, forming so-called
   Track Legs, and joined with strict Segments.  The Legs are expressed
   in RPL Non-Storing Modes and require an encapsulation to add a Source
   Route Header, whereas the Segments are expressed in Storing Mode.  A
   Serial Track comprises a single Leg and provides only one path
   between Ingress and Egress.  A Projected Route signals either a Track
   Leg or a Segment, using extended RPL DAO messages, and a Track is
   installed and maintained with a collection of the so-called Projected
   DAOs.


   But to provide redundancy, a Track must be more complex, either as a
   collection of Legs that may be parallel or cross at certain points,
   or as a more generic DODAG that is rooted at the Ingress Node.  This
   specification introduces the Via Information Option to signal a
   sequence of hops in a Leg or a Segment.

           CPF               CPF          CPF                 CPF

                          Southbound API

      _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-
    _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-

                         +----------+
                         | RPL Root |
                         +----------+
                           (      )
                 (                                  )
           (              DODAG                              )
             (                                           )
     (                                                         )
                                                                     )
          Leg 1              B,                              E
     <-- Segment 1 A,B ---> <------ Segment 2 C,D,E ------>

                FWD  --z  Relay --z   FWD  --z   FWD          Target 1
            z-- Node  z--  Node  z--  Node  z--  Node --z     /
         --z    (A)        (B) \      (C)        (D)  z--    /
   Track                        \                       Track
   Ingress                    Segment 5                 Egress - Tgt 2
     (I)                           \                     (E)
         --z                        \                 z--    \
          z-- FWD   --z  FWD  --z  Relay --z  FWD  --z        \
              Node   z-- Node  z-- Node  z-- Node             Target 3
              (F)        (G)       (H)       (J)

     <------ Segment 3 F,G,H ------> <--- Segment 4 J,E --->
             Leg 2                  H,                      E

     <--- Segment 1 A,B ---> <- S5-> <--- Segment 4 J,E --->
             Leg 3          B,      H,                      E
                                                                     )
      (
                 (                                        )

                       Figure 1: Segments and Tracks

   DetNet and RAW leverage a Controller Plane Function (CPF) called the
   Path Computation Element (PCE) [RFC4655] that interacts with RAW
   Nodes over a Southbound API to set up Tracks.  With this
   specification, the PCE interacts with the Root and the Root
   translates that signaling into RPL signaling.  The southbound
   interface is out of scope.  Forwarding Nodes only understand the
   simple 1-to-1 forwarding sublayer transport operation along a segment
   whereas the more sophisticated Relay nodes can also provide service
   sublayer functions such as Replication and Elimination.  One possible
   mapping with this specification is to signal the Relay Nodes as the
   hops of a Leg and the forwarding Nodes as the hops in a Segment that
   joins the Relays as illustrated in Figure 1.

   Note that while this specification enables to build both Segments
   inside a Leg (aka East-West), such as Segment 2 above which is within
   Leg 1, and Inter-Leg Segments (aka North-South), such as Segment 2
   above which joins Leg 1 and Leg 2, this specification does not signal
   to the Ingress which Inter-Leg Segments are available, so the use of
   North-South Segments and associated PAREO functions will require
   additional specification.  With this work, the only possibility is to
   define overlapping Legs as illustrated in Figure 1, with Leg 3 that
   is congruent with Leg 1 till node B and congruent with Leg 2 from
   node H on, abstracting Segment 5 as an East-West Segment.

> I only performed very opportunistic spell check, please do one yourself.
> Please run idnits check too, it complains about the abstract for me.
> 
> Appended draft with numbered lines and non-numbered line feedback/comments.
>
> Cheers & thanks again
>     Toerless
> 
> 
>      5	ROLL                                                     P.
> Thubert, Ed.
>      6	Internet-Draft
> Cisco Systems
>      7	Updates: 6554 (if approved)
> R.A. Jadhav
>      8	Intended status: Standards Track
> Huawei Tech
>      9	Expires: 28 January 2022                                     M.
> Gillmore
>     10
> Itron
>     11	                                                            27
> July 2021
>     12
>     13
>     14	                  Root initiated routing state in RPL
>     15	                   draft-ietf-roll-dao-projection-19
> ...
> 
>    135	1.  Introduction
>    136
>    137	   RPL, the "Routing Protocol for Low Power and Lossy Networks"
> [RPL]
>    138	   (LLNs), is a generic Distance Vector protocol that is well
> suited for
>    139	   application in a variety of low energy Internet of Things
> (IoT)
>    140	   networks.  RPL forms Destination Oriented Directed Acyclic
> Graphs
>    141	   (DODAGs) in which the Root often acts as the Border Router
> to connect
>    142	   the RPL domain to the Internet.  The Root is responsible to
> select
>                           ^^^^^^^^^^^^^^^
> Really ? What is the "often" case where e.g.: at least nodes in the RPL
> network
> can reach nodes across the Internet ? Aka: from my understanding, the
> common
> deployment would attach to other limited domain (RFC8799) networks, for
> example
> if i correctly glean from rfc9030, but not "the Internet". Otherwise,
> please
>  provide a reference to justify "often ... to the Internet".
> 
> Aka: suggest to replace Internet with "other network" or maybe "a
> backbone".

Makes sense. A backbone is fine.

> 
>    142	   the RPL domain to the Internet.  The Root is responsible to
> select
>    143	   the RPL Instance that is used to forward a packet coming
> from the
>    144	   Internet into the RPL domain and set the related RPL
> information in
> 
> Same comments about "the Internet".

Ack

> 
>    147	   The "6TiSCH Architecture" [6TiSCH-ARCHI] also leverages the
>    148	   "Deterministic Networking Architecture" [RFC8655]
> centralized model
>    149	   whereby the device resources and capabilities are exposed to
> an
>    150	   external controller which installs routing states into the
> network
>    151	   based on some objective functions that reside in that
> external
>    152	   entity.  With DetNet and 6TiSCH, the component of the
> controller that
> 
> The DetNet architecture has a lot more aspects for which i am not sure,
> that
> 6TiSCH matches them (have not gone fully through 6TiSCH-ARCHI). Aka: DetNet
> as
> of today implies the per-hop,per-flow IntServ architecture, but no DiffServ
> for DetNet service hops. Is this what 6TiSCH-ARCHI does as well ?

The sentence above means to say that we leverage the centralized model, not necessarily apply all DetNet concepts at all times.
6TiSCH ARCHI did not describe a service layer per se though the functionality was assumed as multiple in / multiple out time slots, regardless of who is speaking or listening.
A 6TiSCH Track Segment MUST be strict as it matches DetNet forwarding sublayer; and it make sense in RAW cases to associate the source routed hops of a Track Leg to the (service layer) relays.
As it goes, 6TiSCH inherits from RAW which inherits from DetNet and yes the architecture would be the same in my mind. But not necessarily with the goal to enforcing deterministic properties (like bounded latency) all the time. The text I added for your earlier point in termminology covers that I hope.

> 
> If not, then maybe change to "levereges the aspects of the ..." to make
> the claim more scoped to what the paragraph describes.

Changed to 


   The "6TiSCH Architecture" [6TiSCH-ARCHI] leverages a centralized
   model similar to that of "Deterministic Networking Architecture"
   [RFC8655], whereby the device resources and capabilities are exposed
   to an external controller which installs routing states into the
   network based on its own objective functions that reside in that
   external entity.  With DetNet and 6TiSCH, the component of the
   controller that is responsible of computing routes is called a Path
   Computation Element ([PCE]), and the objective functions are
   described in [RFC4655] and encoded in the PCE Protocol (PCEP) by
   [RFC5551].


> 
>    152	   entity.  With DetNet and 6TiSCH, the component of the
> controller that
>    153	   is responsible of computing routes is called a Path
> Computation
>    154	   Element ([PCE]).
>    155
>    156	   Based on heuristics of usage, path length, and knowledge of
> device
>    157	   capacity and available resources such as battery levels and
>    158	   reservable buffers, the PCE with a global visibility on the
> system
>    159	   can compute direct Peer to Peer (P2P) routes that are
> optimized for
>    160	   the needs expressed by an objective function.  This document
> 
> "Based on heuristics" has one doubt whether the result is deerministic. Not
> saying heuristics may not play a role, but maybe fnd a better wording.

The idea is that the controller sees reliability stats and based on those stats it computes how many parallel paths are needed and how many copies of the packets need to be sent to meet the SLA.
The point about DetNet is not to make necessarily deterministic paths but to use a similar model with a controller with "God's view".
Now there can be AI in the controller and if so, it turns the statistics / time series in heuristics. Open to change but there may be loss of information.
So I guess that the language is correct, open to more discussion.



> 
> "global visibility" sounds like weather satellites. More specifically when
> reading, i wonder if the scope of the discussion in this document can be
> constraint to a single RPL network, or if it is important to understand
> that the mechanisms of this document a also enabling more "interesting"
> paths.

As Anand pointed out, the metrics that are reported to the controller and how that happens is out of scope. Added text on that for his review, will be part of the same diff set from v19 to 20.
But yes, the ultimate goal is as you say, basically TE / DetNet paths.




> 
> Aka: If the following is one of the cases for which this drafts work is
> in support of, then i would really like to see this type of picture and
> explanation added as something like a reference picture with explanation:
> 
>                Client11               Client21             ClientN1
>                 |                      |                    |
>   PCE        RPL-network1         RPL-network2         RPL-networkN
>    |        Root1     Rtr1b      Root2    Rtr2b  ...   RootN   RtrNb
>    |          |         |          |        |           |       |
>   ..................................................................
>                        Backbone Network, L2 / L3, e.g.:
>   -+----------+---------+----------+--------+-----------+-------+----

What is rtr1B ? backup for Root? It seems connected to the RPL network; is it? if so, that's not in scope but we could add it to the thought process.

The global network may be as you say but the scope of this doc is within one DODAG.
Continuing a track across another DODAG is feasible but out of scope. Again t keep simple for now.
And again if you have a string use case, the group may change its mind.


> 
> Aka: The PCE can calculate paths between clients in this internetwork.
> An example path between Client11 and ClientN1 is composed of the
> projected path between Client11 and one of the nodes connecting RPL-
> network1
> to the backbone, e.g.: Rtr1b, a path through the backbone, and finally
> a path through RPL-networkN, again including the choice of edge
> router to the backbone, e.g.: RtrNb.

While feasible, this is a story way beyond our spec. For doing the above, we'd not build a Track, but use plain RPL as it is already the shortest path.
Though we use a controller the spec is only a first step towards RAW / DetNet for routing. All the metrics to the root, the policies to select which flow goes in which Track, the schedules or whatever, need to come from RAW/6TiSCH_2.0.
If you need something like this for ANIMA, let's chat outside this thread.

> 
> I guess if the solution does not support RtrIb, then its explanation
> could more easily be constrained to just a single RPL-network.

With RPL the Root is the only connection to the "backbone". That's why it is usually the same box as the 6LoWPAN 6LBR.

> 
>    160	   the needs expressed by an objective function.  This document
> 
> This statement is confusing.
> Without this draft, the objective functions result in specific paths,
> and these paths are insufficient, so the PCE also uses this drafts
> mechanism to achieve better paths. So the objective function clearly
> was not sufficient. Besides, the objective function is very specific
> to parameters used in distributed path calculation, not PCE.

The PCE has its own objective functions, see RFC 4655, that are encoded in PCEP by RFC 5551. 
The goals of a Track are different from those oif the lain DIDAG so the OF will be different. Here we are talking about the Track OF as seen by the PCE. 
Let me add a reference. With the above discussion this becomes:

  The "6TiSCH Architecture" [6TiSCH-ARCHI] leverages a centralized
   model similar to that of "Deterministic Networking Architecture"
   [RFC8655], whereby the device resources and capabilities are exposed
   to an external controller which installs routing states into the
   network based on its own objective functions that reside in that
   external entity.  With DetNet and 6TiSCH, the component of the
   controller that is responsible of computing routes is called a Path
   Computation Element ([PCE]), and the objective functions are
   described in [RFC4655] and encoded in the PCE Protocol (PCEP) by
   [RFC5551].
> 
> In my mind, there is some goal/objective for the path to be calculated,
> and this objective is achieved by OF to create DODAG and paths, and if
> those paths are insufficient, then
> and a controller can achieve that objective by defining a
> but when the controller determinses that it can not solely be achieved
> through objective function parameters, then it starts using this drafts
> mechanism...

Well, think that in non-storing mode all packets are routed through the Root. Very inefficient.
So the function N1 of the draft is to do shortcuts with specific SLAs that will be expressed as PCE OF (not RPL OF though it's all very abstract anyway).
The function N2, of value only for non-storing, is to make the RH smaller for routes in the main DODAG, using storing mode Segments to join the loose SRH.



> 
>    160	   the needs expressed by an objective function.  This document
> 
> Suggest new paragraph before "This"
> 
>    161	   specifies protocol extensions to RPL [RPL] that enable the
> Root of a
>    162	   main DODAG to install centrally-computed routes inside the
> DODAG on
>    163	   behalf of a PCE.
>    172
>    173	   This specification expects that the main RPL Instance is
> operated in
>    174	   RPL Non-Storing Mode of Operation (MOP) to sustain the
> exchanges with
>    175	   the Root.  In that Mode, the Root has enough information to
> build a
>                     ^ Only ?
>    176	   basic DODAG topology based on parents and children, but
> lacks the
>                                                                   ^ still
>    177	   knowledge of siblings.  This document adds the capability
> for nodes
> 
> Aka: in storing mode the root would even have less information, right ?

Exactly; in NS we have the DODAG at the root already. See my point above


> 
>    178	   to advertise sibling information in order to improve the
> topological
>    179	   awareness of the Root.
>    180
>    181	   As opposed to the classical RPL operations where routes are
> injected
>    182	   by the Target nodes, the protocol extensions enable the Root
> of a
>    183	   DODAG to project the routes that are needed onto the nodes
> where they
>    184	   should be installed.  This specification uses the term
> Projected
>    185	   Route to refer to those routes.  Projected Routes can be
> used to
>    186	   reduce the size of the source routing headers with loose
> source
>    187	   routing operations down the main RPL DODAG.  Projected
> Routes can
>    188	   also be used to build transversal routes for route
> optimization and
>    189	   Traffic Engineering purposes, between nodes of the DODAG.
>    190
>    191	   A Projected Route may be installed in either Storing and
> Non-Storing
>    192	   Mode, potentially resulting in hybrid situations where the
> Mode of
>    193	   the Projected Route is different from that of the main RPL
> Instance.
>    194	   A Projected Route may be a stand-alone end-to-end path or a
> Segment
>    195	   in a more complex forwarding graph called a Track.
>    197	   The concept of a Track was introduced in the 6TiSCH
> architecture, as
>    198	   a potentially complex path with redundant forwarding
> solutions along
>    199	   the way.
> 
> Please add a reference to the spec that introduced Track.

Done, that's RFC 9030 

> 
> Where is "stand-alone" introduced ? If elswhere, add reference, otherwise
> say its introduced here. Also, everything else is capitalized, maybe you
> want
> to capitalize also Stand Alone.

Yes that term could be useful. Adding to terminology

> 
>    199     With this specification, a Track is a DODAG formed by a RPL
>    200	   local Instance that is rooted at the Track Ingress.  If
> there is a
>    201	   single Track Egress, then the Track is reversible to form
> another
>    202	   DODAG by reversing the direction of each edge.  A node at
> the ingress
>    203	   of more than one Segment in a Track may use one or more of
> these
>    204	   Segments to forward a packet inside the Track.
> 
> The last sentence is using the term Segment to explain something new, but
> the
> term Segment was not sufficiently defined in before. Other than that a
> Segment
> can be or is always (?) a Projected Route, but Projected Route is also
> undefined.

This is now all introduced in the new Terminology section.



> 
>    206	   The "Reliable and Available Wireless (RAW)
> Architecture/Framework"
>    207	   [RAW-ARCHI] defines the Path Selection Engine (PSE) that
> adapts the
>    208	   use of the path redundancy within a Track to defeat the
> diverse
>    209	   causes of packet loss.
>    210
>    211	   The PSE is a dataplane extension of the PCE; it controls the
>    212	   forwarding operation of the packets within a Track, using
> Packet ARQ,
>    213	   Replication, Elimination, and Overhearing (PAREO) functions
> over the
>    214	   Track segments, to provide a dynamic balance between the
> reliability
>    215	   and availability requirements of the flows and the need to
> conserve
>    216	   energy and spectrum.
>    217
>    218	   The time scale at which the PCE (re)computes the Track can
> be long,
>    219	   using long-term statistical metrics to perform global
> optimizations
>    220	   at the scale of the whole network.  Conversely, the PSE
> makes
>    229	   forwarding decisions at the time scale of one or a small
> collection
>    230	   of packets, based on a knowledge that is limited in scope to
> the
>    231	   Track itself, so it can be refreshed at a fast pace.
> 
> I would suggest to restructure this. Before going into the RPL details as
> you do
> in before, maybe start with the problem to be solved. Which seems to be a
> mechanism to support packet replication, traversal over alternative paths
> and
> duplicate elimination (and seemingly more). 6TiSCH introduced Tracks as the
> concept for redundant paths, RAW inroduced PAREO. This document introduces
> one solution to to build Tracks with RPL und a mechanism called Projected
> Routes.
> 


Agreed, did that in the new Terminology section too. Note that PAREO is not signaled in this spec, new text on that too.


> Something like that to have a more logical sequence of explanations, top to
> bottom.
> 

Ack

>    233	   Projected Routes must be used with the parsimony to limit
> the amount
> 
> Parsimony in he sense you use it, was never used in RFCs, rc871 used law of
> parsimony.
> Suggest to reword for easier reading. For example, "Care must be taken that
> state installed for
> projected routes fit in each devices resources".

What about

   Projected Routes require resources in the routers; the amount of
   state that is installed in each node must be computed to fit within
   the node's memory, and the amount of rerouted traffic must fit within
   the capabilities of the transmission links.  The methods used to
   learn the node capabilities and the resources that are available in
   the devices and in the network are out of scope for this document.
   The method to report the LLN link capacity and reliability statistics
   are also out of scope.  They may be fetched from the nodes through
   network management functions or other forms of telemetry such as OAM.


> 
> Is the amount of traffic solely a responsibility of the installation of
> projected
> routes ? Isn't this also responsibility of the PSE ? (just guessing,
> haven't read
> up on it).

You're quite right, but all that discussion is out of scope; as far as this 
Doc is concerned, there's no ingress policy and not associated PREOF intent.
All the spec can expect in implicitly classical ECMP over redundant paths.
Text added in the introduction covers that, see a little below in this review.

> 
>    234	   of state that is installed in each device to fit within the
> device
>    235	   resources, and to maintain the amount of rerouted traffic
> within the
>    237	   capabilities of the transmission links.  The methods used to
> learn
>    237	   the node capabilities and the resources that are available
> in the
>    238	   devices and in the network are out of scope for this
> document.
>    239
>    240	   This specification uses the RPL Root as a proxy to the PCE.
> The PCE
>    241	   may be collocated with the Root, or may reside in an
> external
>    242	   Controller.
>    243
>    244	   In that case, the PCE exchanges control messages with the
> Root over a
>               ^^^^ the latter
>    245	   Southbound API that is out of scope for this specification.
> The
>                      ^ PCE
>    246	   algorithm to compute the paths and the protocol used by an
> external
>    247	   PCE to obtain the topology of the network from the Root are
> also out
>    248	   of scope.
> 
> In his AD review for draft-ietf-bier-te-arch, Alvaro asked me to provide
> examples
> for such topology discovery. YOu can of course wait out if this will happen
> for
> his doc too, i personally have no strong opinions, but if you need example
> text, see draft-ietf-bier-te-arch, 3.2.1.1, Third paragraph.

Well, that could be the same RPL formats, or anything else. 

The section would become:


   There specification considers that there's a unique PCE with full
   view of the network.  Distributing the function for a large network
   is out of scope.  This specification uses the RPL Root as a proxy to
   the PCE.  The PCE may be collocated with the Root, or may reside in
   an external Controller.  In that case, the protocol between the Root
   and the PCE is out of scope and abstracted by / mapped to RPL inside
   the DODAG; one possibility is for the Root to transmit the RPL DAOs
   with the SIOs that detail the parent/child and sibling information.


> 
> Personally, i do not like the conditioning that a southbound API is
> supposedly
> only necessary when the PCE is external from the RPL Root. IMHO, there
> always
> needs to be at least an informat model for Projected Routes. Which you may
> agree on, but its not mentioned. PCE and RPL Root can just be independent
> software entities on the same node.

Very true; we'll need a spec for that

> 
> 
>    260	2.2.  Glossary
>    261
>    262	   This document often uses the following acronyms:
>    263
>    264	   CMO:  Control Message Option
>    265	   DAO:  Destination Advertisement Object
>    266	   DAG:  Directed Acyclic Graph
>    267	   DODAG:  Destination-Oriented Directed Acyclic Graph; A DAG
> with only
>    268	      one vertex (i.e., node) that has no outgoing edge (i.e.,
> link)
>    269	   LLN:  Low-Power and Lossy Network
>    270	   MOP:  RPL Mode of Operation
>    271	   P-DAO:  Projected DAO
>    272	   P-Route:  Projected Route
>    273	   PDR:  P-DAO Request
>    274	   RAN:  RPL-Aware Node (either a RPL Router or a RPL-Aware
> Leaf)
>    275	   RAL:  RPL-Aware Leaf
>    276	   RH:  Routing Header
>    285	   RPI:  RPL Packet Information
>    286	   RTO:  RPL Target Option
>    287	   RUL:  RPL-Unaware Leaf
>    288	   SIO:  RPL Sibling Information Option
>    289	   SR-VIO:  A Source-Routed Via Information Option, used in
> Non-Storing-
>    290	      Mode P-DAO messages.
>    291	   TIO:  RPL Transit Information Option
>    292	   SF-VIO:  A Via Information Option, used in Storing-Mode P-
> DAO
>    293	      messages.
>    294	   VIO:  A Via Information Option; it can be a SF-VIO or an SR-
> VIO.
> 
> I had a hard time remembering SR-VIO and SF-VIO throughout the text. Maybe
> SM-VIO and NSM-VIO would be easier to remember and better because they
> explicitly
> refer to the explanation (Storing Mode, Non-Storing Mode).

Adopted

> 
> I'll spare you the request for a full Terminology section for these terms
> with
> references, maybe others will do so.


Did it

> 
> It would make sense though to move the terms newly introduced in this doc,
> such as P-Route , P-DAO and the like into section 2.3 and rename 2.3 to
> "New terms".

They are not necessarily new; a lot is copied from 6TiSCH or RAW.

> Derived terms like P-Route can of course stay concise.
> 
>    296	2.3.  Other Terms
> ...
> 
>    327	3.  Extending RFC 6550
> 
> You have tagged this draft as an Update to RFC6550. You should IMHO have a
> section where you summarize what that exactly entails, ideally call
> "Updated to RFC6550".
> 
> AFAIK, we do not have a good formal structure for such update description,
> Mirja
> tried to propose some more keywords, but so far, Update can mean anything.
> 

From the latest understanding I gather that we are extending but not updating, no impact on existing code.
So I removed the tag. Alvaro will help here I guess.

> I would suggest to summarize what type of update this is by making
> statements
> answering the following type of questions (maybe i forgot some). These
> statements
> are also useful when they are answering the questions below with No. I
> leave it up to
> you if you feel this would be good to go here in the beginnin, or if you'd
> rather
> want to summarize it in the end of the document.

We have 

   3.  Extending RFC 6550  . . . . . . . . . . . . . . . . . . . . .  12
   4.  Extending RFC 6553  . . . . . . . . . . . . . . . . . . . . .  15
   5.  Extending RFC 8138


> 
> Questions:
> 

Very useful list!

> Does this document specify any functionality that is REQUIREDS or
> RECOMMENDED
> for implementations of RFC6550 ? If so, please detail them.  Else, this can
> be called
> a fully OPTIONAL update for RFC6550.

It is optional. This can operate in a mixed network.


> Will implementations supporting this RFC continue to be backward compatible
> with
> all prior RFCs, RFC6550 and related ? If not, please describe any changes
> in
> interoperability.

They will

> 
> Does this RFC introduce any functional extensions to RFC6550 or other RFC
> through mechanisms other than pre-defined extension points ?
> 
> Are any pre-defined extension points of RFC6550 or other RFC used, which
> are not
> already IANA registries ?  If so, please include references to the original
> RFC
> section where that extension point was described.
> 

No such thing

> (first time i am trying to formalize these type of question...)

We need to store that somewhere, very useful

> 
> IMHO outside the scope of documenting the "Update" flag is mixed
> deployment.
> If there is no text for that, would be good to add that somehwere. Update
> section
> wouldn't be a bad place.
> 

I removed the update tag

> 
>    341	3.1.  Projected DAO
> 
> At this point i am worried whether i am correctly guessing at the
> forwarding
> plane behavior of projected routes, because i felt it was presented in the
> Intro section mostly en-passant, but it is hard to follow the document with
> the correct explanation of how forwarding works.
> 
> What i figured from the Intro is that Projected Routes would allow the root
> to specify a loose path in the RH, even though only non-storing mode is
> used.
> Whenever a RPL node then sees a next-op in the RH it looks for a projected
> Route towards that next-hop and hmm... just forwards the packet to the
> first next hop on the projeted route ? Or it needs to change the routing
> header to prepend the projected route ?

Makes sense, all this discussion on PSE is really extraneous. I changed the intro to


   The "Reliable and Available Wireless (RAW) Architecture/Framework"
   [RAW-ARCHI] extends the definition of Track, as being composed to
   east-west directional segments and north-south bidirectional
   segments.  It also defines the Path Selection Engine (PSE) that
   adapts the use of the path redundancy within a Track to defeat the
   diverse causes of packet loss.

   This specification prepares for RAW by setting up the Tracks, but
   does not provide ingress policy that selects which packets are routed
   through which Track, and how the PSE may use the path redundancy
   within the Track.  As far as this specification is concerned, the
   main DODAG provides a default route via the Root, and the Tracks
   provide mode specific routes to the Track targets.  The use of the
   redundancy is limited to ECMP load balancing, and all the segments
   are east-west unidirectional only.


> 
> And i have absolutely no idea what interactions there are between PSE and
> Projected routes in the forwarding plane..

I hope the text above clarifies

> 
> Section 9 has some signalling examples. Maybe it would be good to create
> some introductory example that shows the relevant forwarding plane aspects.
> 
>    343	   Section 6 of [RPL] introduces the RPL Control Message
> Options (CMO),
>    344	   including the RPL Target Option (RTO) and Transit
> Information Option
>    345	   (TIO), which can be placed in RPL messages such as the
> Destination
>    346	   Advertisement Object (DAO).
> 
>    346	   This specification extends the DAO
>    347	   message with the Projected DAO (P-DAO); a P-DAO message
> signals a
>    348	   Projected Route to one or more Targets using the new CMOs
> presented
>    349	   therein.
> 
> This is confusing. If i am guessing right, i would suggest to write:


> 
> This specification defines a new CMO carrying a Projected Route. When this
> Projected Route CMO is included in a DAO message, this is called a
> Projected DAO (P-DAO).

Used the proposed text with a twist; many thanks!

   This specification defines a new CMO carrying a Projected Route.
   A DAO message that conveys the new CMO is called a Projected DAO (P-DAO).


> 
> I do not get the "signals _a_ Projected Route to one or _more_ Targets".
> How can
> a single P-DAO be signalled to more than one recipient ?	

Storing Mode P-DAO flows from egress to ingress, installing the segment on the way ala intserv / rsvp.

> 
> 
>    349     This specification enables to combine one or more Projected
>    350	   Routes into a DODAG called a Track, that is traversed to
> reach the
>    351	   Targets.
>    352
>    353	   The Track is assimilated with the DODAG formed for a Local
> RPL
>    354	   Instance.  The local RPLInstanceID of the Track is called
> the
>    355	   TrackID, more in Section 7.2.  A P-DAO message for a Track
> signals
>    356	   the TrackID in the RPLInstanceID field.  The Track Ingress
> is
>    357	   signaled in the DODAGID field of the Projected DAO Base
> Object; that
>    358	   field is elided in the case of the main RPL Instance.  The
> Track
>    359	   Ingress is the Root of the Track, as shown in Figure 1.
> 
> How about the text here first finishes up the specification of the P-DAO
> CMO
> before explaining some, but likely not all details of how this stuff
> works together ?


This is now described in the terminology section, with illustration.

> 
>    360
>    361	   This specification defines the new "Projected DAO" (P) flag.
> The 'P'
>    362	   flag is encoded in bit position 2 (to be confirmed by IANA)
> of the
>    363	   Flags field in the DAO Base Object.  The Root MUST set it to
> 1 in a
>    364	   Projected DAO message.  Otherwise it MUST be set to 0.  It
> is set to
>    365	   0 in legacy implementations as specified respectively in
> Sections
>    366	   20.11 and 6.4 of [RPL]. .
>    367
>    368	      0                   1                   2
> 3
>    369	      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
> 9 0 1
>    370	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    371	     |    TrackID    |K|D|P|  Flags  |   Reserved    |
> DAOSequence   |
>    372	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    373	     |
> |
>    374	     +
> +
>    375	     |
> |
>    376	     +               IPv6 Address of the Track Ingress
> +
>    377	     |
> |
>    378	     +
> +
>    379	     |
> |
>    380	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    381	     |   Option(s)...
>    382	     +-+-+-+-+-+-+-+-+
>    383
>    384	                    Figure 1: Projected DAO Base Object
>    385
>    386	   New fields:
>    387
>    388	   TrackID:  In the case of a P-DAO, the RPLInstanceID field is
> called
>    397	      TrackID.  This is a naming convenience but does not
> change the
>    398	      semantics and format of the RPLInstanceID that is used as
> TrackID.
> 
> Maybe then stick to calling it RPLInstanceID in the ASCII and not calling
> it
> a new field (nitpicking).

I must stick the the Archi (RAW, 6TiSCH) ; There could be other forms of TrackIDs.
Changing "called TrackID" to "used as TrackID"

> 
>    400	   P:  1-bit flag (position to be confirmed by IANA).
>    401
>    402	      The 'P' flag is set to 1 by the Root to signal a
> Projected DAO,
>    403	      and it is set to 0 otherwise.
>    404
>    405	   In RPL Non-Storing Mode, the TIO and RTO are combined in a
> DAO
>    406	   message to inform the DODAG Root of all the edges in the
> DODAG, which
>    407	   are formed by the directed parent-child relationships.
> Options may
>    408	   be factorized; multiple RTOs may be present to signal a
> collection of
>    409	   children that can be reached via the parent(s) indicated in
> the
>    410	   TIO(s) that follows the RTOs.  This specification
> generalizes the
>    411	   case of a parent that can be used to reach a child with that
> of a
>    412	   whole Track through which both children and siblings of the
> Track
>    413	   Egress are reachable.
> 
> This again seems to be mixing larger logical behavior explanation into what
> probably should be just the spec for the new message encodings here. See
> how
> you can decouple the larger system explanations from the encoding spec.

True enough but that's 2 sentences that may help the reader there. 
Let me add a fwd ref to 7.3.x where the details are, as below:



   Like in a DAO message, the RTOs can be factorized in a P-DAO, but the
   Via Information Options cannot.  A P-DAO contains one or more RTOs
   that indicate the destinations that can be reached via the Track, and
   exactly one VIO that signals a sequence of nodes.  In Non-Storing
   Mode, the Root sends the P-DAO to the Track Ingress where the source-
   routing state is stored, whereas in Storing Mode, the P-DAO is sent
   to the Track Egress and forwarded along the Segment in the reverse
   direction, installing a Storing Mode state to the Track Egress at
   each hop, see Section 7.3.2 and Section 7.3.1 respectively.  In both
   cases the Track Ingress is the owner of the Track, and it generates
   the P-DAO-ACK when the installation is successful.


> 
>    415	   New CMOs called the Via Information Options (VIO) are
> introduced for
>    416	   use in P-DAO messages as a multihop alternative to the TIO.
> One VIO
> 
> a) Add forward pointer to section defining VIO
> 



> b I get the feeling as if the whole mentioning of TIO and RTO above should
> simply
> go in a separate chapter for RPL experts with a title like "Why are TIO and
> RTO
> not sufficient". Or "How to combine TIO/RTO with SF-VIO" etc.. but try to
> stick to topic at hand here. Feeble minded readers like myself can only
> balance
> a limited number of partially understood terms in the air at the same time.

Moved stuff around, the section is now simpler and focused on P DAO formats 

> 
>    417	   is the Stateful VIO (SF-VIO); the SF-VIO installs Storing-
> Mode
>                                                             ^ a ?
>    418	   Projected Route  along a strict segment.  The other is the
> Source-
>                           ^ s ?
>    419	   Routed VIO (SR-VIO); the SR-VIO installs a Non-Storing-Mode
> Projected
>    420	   Route at the Track Ingress, which uses that state to
> encapsulate a
>    421	   packet with a Routing Header (RH) to the Track Egress.
> 
> Question... If i just use basic RPL with non-storing mode, i already
> need an RH. And that is let's say a strict path. So after the first hop of
> that
> RH, my next hop is actually a track ingres for which i have SR-VIO. Does
> this
> mean i need to do a full new IPv6 encap of the packet to insert the new
> RH for the track ? 

Added a fwd link to 7.6.  Forwarding Along a Track
I guess you later found the detailed answer in for your question in  9.1.1.  Stitched Segments
The RH for the main DODAG becomes loose, and SM-VIOs connect the dots. No reencaps unless you really like them.




> If so:
> 
> a) i'd say: "to encapsulate the packet into an IPv6 packe with a new RH to
> the track egress.

done


> b) What about MTU issues due to the encap ? Can not find word MTU in text.


Was true. 7.3 now has 


   A Non-Storing-Mode P-DAO signals a strict or loose sequence of nodes
   between the Track Ingress (excluded) and a Track Egress (included).
   It installs a source-routed path of a higher precedence within the
   Track indicated by the P-DAO Base Object, towards the Targets
   indicated in the Target Options.  The source-routed path requires a
   Source-Routing header which implies an encapsulation to add the SRH
   to an existing packet.  In order to avoid fragmentation, e.g., using
   [6LoWPAN], [RFC8930], and/or [RFC8931] in a 6LoWPAN LLN, it is
   RECOMMENDED to allow an MTU that is larger than 1280 in the main
   DODAG and allows for the additional headers while exposing only 1280
   to the 6LoWPAN Nodes as indicated by section 4 of [6LoWPAN].


> 
>    423	   Like in a DAO message, the RTOs can be factorized in a P-
> DAO, but the
>    424	   Via Information Options cannot.
> 
>    424	   A P-DAO contains one or more RTOs
>                   ^ MUST/SHOULD/MAY ?
> 

This is said later, in uppercase as below; adding a fwd link


    A P-DAO message MUST contain exactly one VIO, which is either a SM-VIO or an
    NSM-VIO, and MUST follow one or more RTOs.


>    425	   that indicate the destinations that can be reached via the
> Track, and
>    426	   exactly one VIO that signals a sequence of nodes .  In Non-
> Storing
>                                                            ^ That form the
> track ?

Now


   A P-DAO contains one or more RTOs to indicate the target
   (destinations) that can be reached via the P-Route, and at most one
   VIO that signals the sequence of nodes to be followed, more in
   Section 7.  


> 
>        root ....... Track-Ingres ...(Track Midpoints)... Track-Egres .....
> Destination1
>                                                                      ...
>                                                                      ...
> DestinationN
> P-DAO has:
>   one RTO for each DestinationI
>   one VIO for the Track: Midpoints,Egres
> 
> Semantic: When packet reaches Track-Ingres, it checks whether destination
> or
> next-hop is one of the DestinationI, if so, then use the track. If SR-VIO,
> then encap IPv6/RH, if SF-VIO, then just forward to first hop on track,
> which will also have installed the right state for the track...

True, that's basic routing since the Track provides a more specific route, and SR VIO (now NSM VIO) signals a tunnel.

> 
> Something like this with picture would make the whole concept so much
> easier to understand (to me).

There's now one with 2 legs in the terminology. And then the whole section 9.

> 
>    427	   Mode, the Root sends the P-DAO to the Track Ingress where
> the source-
>    428	   routing state is stored.  In Storing Mode, the P-DAO is sent
> to the
>    429	   Track Egress and forwarded along the Segment in the reverse
>    430	   direction, installing a Storing Mode state to the Track
> Egress at
>    431	   each hop.  In both cases the Track Ingress is the owner of
> the Track,
>    432	   and it generates the P-DAO-ACK when the installation is
> successful.
> 
> How would the root know whether in storing node all the track midpoints
> did successfully install the necessary track state ? If not, hen it would
> be
> worth pointing out how storing mode may end up being less reliable ?

There's an ack sent by the Ingress if OK or any node along the segment if not OK. There's retries. But arguably there's more complexity and more chances to fail. 
I'd rather leave that for the reader to estimate in his case.



> 
>    434	3.2.  Sibling Information Option
> 
> Is there any reason why we wouldn't first want to have the VIO section here
> so a feeble minded reader could pop off some open definitions of the stack
> and declare understanding victory on whats explaind so far ?

Added, the text was there but the section missing

> 
> To answe my own question, i think your structure is around whatever RFC
> is supposedly extended by a new semantic introduced, but that results in
> terrible flow of reading. I would strongly suggest to  reshuffle the
> text from the most simple cases to the most complex cases and have
> separate short sections summarizing for each affected RFC how it is
> extended usin the template questions above, using references to the main
> specification part.

OK, I see you. I moved section 9 up so we see what we're trying to achieve before we learn how that is done

> 
>    436	   This specification adds another CMO called the Sibling
> Information
>    437	   Option (SIO) that is used by a RPL Aware Node (RAN) to
> advertise a
>    438	   selection of its candidate neighbors as siblings to the
> Root, more in
>    439	   Section 6.4.  The sibling selection process is out of scope.
> The
>    440	   expectation is that a node reports a Sibling Address in a
> SIO based
>    441	   on an active address registration [RFC8505] from that
> sibling for
>    442	   that address with the 'R' flag not set in the EARO.  The
> node may
>    443	   assess the liveliness of the sibling at any time by
> performing a
>    444	   registration for one of its own addresses, either a link
> local or a
>    453	   global one, depending on whether the node expects the
> sibling to
>    454	   perform a matching advertisement in its own SIO.
> 
> I am not even going to try to understand this without a prior graphical
> example use case on how this would be used.
> 
>    456	3.3.  P-DAO Request
>    457
>    458	   Two new RPL Control Messages are also introduced, to enable
> a RAN to
>    459	   request the establishment of a Track between self as the
> Track
>    460	   Ingress Node and a Track Egress.  The RAN makes its request
> by
>    461	   sending a new P-DAO Request (PDR) Message to the Root.  The
> Root
>    462	   confirms with a new PDR-ACK message back to the requester
> RAN, see
>    463	   Section 6.1 for more.
> 
> The split between this text and 6.1 is weird and IMHO not helpfull. Merge.
> WHats ven the logic to split some explanations out into section 6 and leave
> others (P-DAO for example) here ?

The logic is imposed on us by the "update rfc-blah" ways. The update is gone but we still need to summarize what we extend.
I'm trying to minimize what goes in the extend "sections", you'll see a lot of reshuffling


> 
> I would also like some motivational example of what would make a RAN send
> out a P-DAO Request. Could it be that the RAN is a PSE ingres ? I am
> asking,
> because this looks like a request where it might be highly desirable to
> have
> more contextual information in the request than just the few fields defined
> in
> 6.1. How about a 32 bit policy-ID field or the like.. But just guessing.
> Something for the PCE to match up PSE stuff it established with P-DAO
> requests it receives later. Unless you feel strongly that association is
> unambiguous now.

I agree with the intention, though this can get quite complex quite quickly so we kept it explicitly out of scope.
I suspect we'll need a new TLV for that sort of control. There could be multiple flows, as many descriptors




> 
>    463     A positive PDR-ACK indicates that the Track
>    464	   was built and that the Roots commits to maintain the Track
> for the
>    465	   negotiated lifetime.  In the case of a complex Track, each
> Segment is
>    466	   maintained independently and asynchronously by the Root,
> with its own
>    467	   lifetime that may be shorter, the same, or longer than that
> of the
>    468	   Track.
> 
> I guess those different liftimes are not meant to make the solution more
> fragile
> by randomnly expiring timers for different parts of a track, but because
> the
> the parts of a projected routes affected can only change when they expire ?
> In any case, an explanation of the semantic impact of lifetimes would be
> useful.

Once you tear down a segment to replace it, everything is desynchronized. 


> 
>    468     The Root may use an asynchronous PDR-ACK with an negative
>    469	   status to indicate that the Track was terminated before its
> time.
> 
> And if that happens, what should a RAN do that has some configuration
> telling
> it to request the P-DAO ? Sounds like the need for the definition of some
> exponential back-off re-request scheme ?
 
I added a neg  status values to indicate transient - retry later
Other values indicates permanent. Pe your other request I move the text to the P DAO RAQ section.
It now looks like




5.1.  New P-DAO Request Control Message

   The P-DAO Request (PDR) message is sent by a Node in the main DODAG
   to the Root.  It is a request to establish or refresh a Track where
   this node is Track Ingress, and signals whether an acknowledgment
   called PDR-ACK is requested or not.  A positive PDR-ACK indicates
   that the Track was built and that the Roots commits to maintain the
   Track for the negotiated lifetime.

   The Root may use an asynchronous PDR-ACK with an negative status to
   indicate that the Track was terminated before its time.  A status of
   "Transient Failure" (see Section 10.9) is an indication that the PDR
   may be retried after a reasonable time that depends on the
   deployment.  Other negative status values indicate a permanent error;
   the tentative must be abandoned until a corrective action is taken at
   the application layer or through network management.




> Also, when the request is just renewed upon timeout, but the reply changes,
> i figure that the impact in stateful mode may be some inconsistent
> forwarding
> while the track midpoints update their routes. No issue i guess in
> stateless.
> Would be useful to mention. Not sure if/what countermeasures for
> consistency
> RPL already provides. If it does, would be good to mention.

This would be guessing. I'm not sure I'd know what to write.



> 
>    471	3.4.  Extending the RPI
>    472
>    473	   Sending a Packet within a RPL Local Instance requires the
> presence of
>    474	   the abstract RPL Packet Information (RPI) described in
> section 11.2.
>    475	   of [RPL] in the outer IPv6 Header chain (see
> [USEofRPLinfo]).  The
>    476	   RPI carries a local RPLInstanceID which, in association with
> either
>    477	   the source or the destination address in the IPv6 Header,
> indicates
>    478	   the RPL Instance that the packet follows.
>    479
>    480	   This specification extends [RPL] to create a new flag that
> signals
>    481	   that a packet is forwarded along a projected route.
>    482
>    483	   Projected-Route 'P':  1-bit flag.  It is set to 1 if this
> packet is
>    484	      sent over a projected route and set to 0 otherwise.
> 
> This leaves me guessing when and how this applied. I can see how this would
> be
> applicable to both stateful and stateless Projected route operastions, but
> in
> either case, the forwarding plane behavior needs to be explained, ideall
> with
> example for each (of the two?) cases.


I clarified that it is set in the RPI in the IPinIP encapsulation and not mutable.

   Projected-Route 'P':  1-bit flag.  It is set to 1 in the RPI that is
      added in the encapsulation when a packet is sent over a Track.  It
      is set to 0 when a packet is forwarded along the main Track,
      including when the packet follows a Segment that joins loose hops
      of the main DODAG.  The flag is not mutable en-route.

   The encoding of the 'P' flag in native format is shown in Section 4.2
   while the compressed format is indicated in Section 4.3.
> 
> Also: Please add forward pointer to next section saying "For concrete
> encoding
> of the P flag, see section 4.".

Sure, see above

> 
>    486	4.  Extending RFC 6553
> 
> I think this RFC MUST claim to be an update to RFC6553, because the P-flag
> added to this concrete RPI RFC exhausts a non-IANA extension point, and
> the only way to formally avoid that any other RFCs could collide (and
> allocate
> the same bit) is to make this RFC an update to RFC6553. Same logic also
> applies to update for RFC6550 and P-DAO P flag.

Raising this to Alvaro's attention. This update thingy is always a tight discussion with him.


> 
> Just to be save, claim also update to RFC9008 to given how you mention it,
> and we obviously want this RFC to also apply to RPI headers with 0x23.
> 
>    511	                                     +-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    512	                                     |  Option Type  |  Opt
> Data Len |
>    513	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    514	     |O|R|F|P|0|0|0|0| RPLInstanceID |          SenderRank
> |
>    515	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    516	     |                         (sub-TLVs)
> |
>    517	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    518
>    519	                    Figure 2: Extended RPL Option Format
>    520
>    521	   Option Type:  0x23 or 0x63, see [USEofRPLinfo]
>    522
>    523	   Opt Data Len:  See [RFC6553]
>                          ^ unchanged,
>    524
>    525	   'O', 'R' and 'F' flags:  See [RFC6553].  Those flags MUST be
> set to 0
>    526	      by the sender and ignored by the receiver if the 'P' flag
> is set.
>    527
>    528	   Projected-Route 'P':  1-bit flag as defined in Section 3.4.
>    529
>    530	   RPLInstanceID:  See [RFC6553].  Indicates the TrackId if the
> 'P' flag
>    531	      is set.
> 
> Add 'see also section 3.1' ?

Sure

> 
>    533	   SenderRank:  See [RFC6553].  This field MUST be set to 0 by
> the
>    534	      sender and ignored by the receiver if the 'P'flag is set.
> 
> Would you mind to add expanation as to what can happen when this passes via
> a router not supporting
> this RFC and therefore ignoring the P flag but interpreting the other
> fields ?

This cannot happen. RPL routers that forward along an instance must understand that instance. 
That was already true with RFC 6550. A Track is actually a RPL instance.


> 
>    536	5.  Extending RFC 8138
>    537
>    538	   Section 6.3 of [RFC8138] presents the formats of the 6LoWPAN
> Routing
>    539	   Header of type 5 (RPI-6LoRH) that compresses the RPI for
> normal RPL
>                                        ^
>    in the IANA "Critical 6LoWPAN Routing Header Type Registry"
> 
>    540	   operation.  The format of the RPI-6LoRH is not suited for
> Projected
>    541	   routes since the O,R,F flags are not used and the Rank is
> unknown and
>    542	   ignored.
>    543
>    544	   This specification introduces a new 6LoRH, the P-RPI-6LoRH,
> with a
>    545	   type of 7.  The P-RPI-6LoRH header is usually a a Critical
> 6LoWPAN
>                     ^                                   ^^^
>                     |
>    in the IANA "Critical 6LoWPAN Routing Header Type Registry"
> 
>    546	   Routing Header, but it can be elective as well if an SRH-
> 6LoRH is
>                                       ^^^^^^^^^^^^^^^^^^^
> also use type TBD1 from the IANA "Electivce 6LoWPAN Routing Header Type
> Registry"
> 
> https://www.iana.org/assignments/_6lowpan-parameters/_6lowpan-
> parameters.xhtml#critical-6lowpan-routing-header-type
> 
> Elective Type 7 is already gone. If you want to get the same numbers for
> elective and critical yo may want to also revisit to choose 7 for critical.
> 
> Who made you be so adventerous to write an actual number 7 into the draft
> without an
> early allocation anyhow ? How about asking for an early allocation ?

Thanks for the catch : ) No need for early allocation. I added (suggested) and moved to 8

> 
>    547	   present and controls the routing decision.
>    548
>    549	   The P-RPI-6LoRH is designed to compress the RPI along RPL
> Projected
>    550	   Routes.  It sformat is as follows:
>                      ^^^
>    551
>    565	                0                   1                   2
>    566	                0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
>    567	               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +
>    568	               |1|0|E| Length  | 6LoRH Type 7  | RPLInstanceID
> |
>    569	               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +
>    570
>    571
>    572	                        Figure 3: P-RPI-6LoRH Format
>    573
>    574	   Elective 'E':  See [RFC8138].  The 'E' flag is set to 1 to
> indicate
>    575	      an Elective 6LoRH, meaning that it can be ignored when
> forwarding.
> 
> Please explain length or point to RFC8138 (inchanged from RFC8318, section
> XX.YY).


Added  in 4.3.  Extending RFC 8138

   The 6LoWPAN Routing Header [RFC8138] specification introduces a new
   IPv6 over Low-Power Wireless Personal Area Network (6LoWPAN)
   [RFC6282] dispatch type for use in 6LoWPAN route-over topologies,
   which initially covers the needs of RPL data packet compression.

   Section 4 of [RFC8138] presents the generic formats of the 6LoWPAN
   Routing Header (6LoRH) with two forms, one Elective that can be
   ignored and skipped when the router does not understand it, and one
   Critical which causes the packet to be dropped when the router cannot
   process it.  The 'E' Flag in the 6LoRH indicates its form.  In order
   to skip the Elective 6LoRHs, their format imposes a fixed expression
   of the size, whereas the size of a Critical 6LoRH may be signaled in
   variable forms to enable additional optimizations.
> 
> Please DO NEVER include the assicned number into the field, just write
> "6LoRH Type".
> Explain in text that is the assined type for criical or elective depending
> on E flag.
> 


Now reads as 

   This specification introduces a new 6LoRH, the P-RPI-6LoRH that can
   be used in either Elective or Critical 6LoRH form, see Table 21 and
   Table 22 respectively.  The new 6LoRH MUST be used as a Critical
   6LoRH, unless an SRH-6LoRH is present and controls the routing
   decision, in which case it MAY be used in Elective form.

   The P-RPI-6LoRH is designed to compress the RPI along RPL P-Routes.
   Its format is as follows:

                0                   1                   2
                0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
               |1|0|E| Length  |  6LoRH Type   | RPLInstanceID |
               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                        Figure 5: P-RPI-6LoRH Format

   Type:  IANA is requested to define the same value of the type for
      both Elective and Critical forms.  A type of 8 is suggested.

   Elective 'E':  See [RFC8138].  The 'E' flag is set to 1 to indicate
      an Elective 6LoRH, meaning that it can be ignored when forwarding.

> Explain that RPLInstanceID is the TrackID (see section 3.1 ?) Or if not,
> what it is.

Done, pointing to the relevant section after all the reshuffling

Looks like 


   TrackID:  8-bit field.  In the context of this specification, the
      TrackID field signals the RPLInstanceID of the DODAG formed by the
      Track, see Section 3.3 and Section 6.2.

> 
>    577	6.  New RPL Control Messages and Options
>    578
>    579	6.1.  New P-DAO Request Control Message
>    580
>    581	   The P-DAO Request (PDR) message is sent by a Node in the
> main DODAG
>    582	   to the Root.  It is a request to establish or refresh a
> Track where
>    583	   this node is Track Ingress.  The source IPv6 address of the
> PDR
>    584	   signals the Track Ingress of the requested Track, and the
> TrackID is
>    585	   indicated in the message itself.  One and only one RPL
> Target Option
>    586	   MUST be present in the message.  The RTO signals the Track
> Egress,
>    587	   more in Section 7.1.
>    588
>    589	   The RPL Control Code for the PDR is 0x09, to be confirmed by
> IANA.
> 
> Correct form is TBDi (TBD2 ?) instead of 0x09.
> If you are not in a hurry, all TBDi can be resolved during IANA/RFC-editor
> procss.
> If you want to lock in a number get an early allocation first.

We want interoperable early implementations. It does not hurt to suggest does it?



> 
>    590	   The format of PDR Base Object is as follows:
>    591
>    592	      0                   1                   2
> 3
>    593	      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
> 9 0 1
>    594	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    595	     |   TrackID     |K|R|   Flags   |  ReqLifetime  |
> PDRSequence   |
>    596	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    597	     |   Option(s)...
>    598	     +-+-+-+-+-+-+-+-+
>    599
>    600	                     Figure 4: New P-DAO Request Format
>    601
>    602	   TrackID:  8-bit field indicating the RPLInstanceID
> associated with
>    603	      the Track.
>    604
>    605	   K:  The 'K' flag is set to indicate that the recipient is
> expected to
>    606	      send a PDR-ACK back.
>    607
>    608	   R:  The 'R' flag is set to request a Complex Track for
> redundancy.
>    609
>    610	   Flags:  Reserved.  The Flags field MUST initialized to zero
> by the
>    611	      sender and MUST be ignored by the receiver
>    612
>    613
>    614
>    615
>    616	Thubert, et al.          Expires 28 January 2022
> [Page 11]
>    617
> 
> 
>    618	Internet-Draft               DAO Projection
> July 2021
>    619
>    620
>    621	   ReqLifetime:  8-bit unsigned integer.  The requested
> lifetime for the
>    622	      Track expressed in Lifetime Units (obtained from the
> DODAG
>    623	      Configuration option).
>    624
>    625	      A PDR with a fresher PDRSequence refreshes the lifetime,
> and a
>    626	      PDRLifetime of 0 indicates that the track should be
> destroyed.
> 
> ... It would be sent for example when the function for which the Node
> requested
> the track is deactivated. ?!

True. Added "                                 e.g., when the
              application that requested the Track terminates. "



> 
>    628	   PDRSequence:  8-bit wrapping sequence number, obeying the
> operation
>    629	      in section 7.2 of [RPL].  The PDRSequence is used to
> correlate a
>    630	      PDR-ACK message with the PDR message that triggered it.
> It is
>    631	      incremented at each PDR message and echoed in the PDR-ACK
> by the
>    632	      Root.
>    633
>    634	6.2.  New PDR-ACK Control Message
>    635
>    636	   The new PDR-ACK is sent as a response to a PDR message with
> the 'K'
>    637	   flag set.  The RPL Control Code for the PDR-ACK is 0x0A, to
> be
>    638	   confirmed by IANA.  Its format is as follows:
> 
> Same IANA/TBD concern as above.

Well yes; same requirement too ; )

> 
>    639
>    640	      0                   1                   2
> 3
>    641	      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
> 9 0 1
>    642	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    643	     |    TrackID    |     Flags     | Track Lifetime|
> PDRSequence  |
>    644	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    645	     | PDR-ACK Status|                Reserved
> |
>    646	     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+
>    647	     |  Option(s)...
>    648	     +-+-+-+-+-+-+-+
>    649
>    650	                Figure 5: New PDR-ACK Control Message Format
>    651
>    652	   TrackID:  The RPLInstanceID of the Track that was created.
> The value
>    653	      of 0x00 is used to when no Track was created.
>                               ^^
> 
> Same question as in before. Any standardized logic what to do in case of
> 0x00 failure,
> if not, then why not.

This was a leftover when any node could create a Track starting at this node.
Now only the ingress can issue a PDR, and it must provide a trackID.

> 
>    654
>    655	   Flags:  Reserved.  The Flags field MUST initialized to zero
> by the
>    656	      sender and MUST be ignored by the receiver
>    657
>    658	   Track Lifetime:  Indicates that remaining Lifetime for the
> Track,
>    659	      expressed in Lifetime Units; the value of zero (0x00)
> indicates
>    660	      that the Track was destroyed or not created.
>    661
>    662	   PDRSequence:  8-bit wrapping sequence number.  It is
> incremented at
>    663	      each PDR message and echoed in the PDR-ACK.
> 
> according to section 7.2 of [RPL] ?!
> 
>    665	   PDR-ACK Status:  8-bit field indicating the completion.  The
> PDR-ACK
>                                                                  ^ status
>    666	      Status is substructured as indicated in Figure 6:
>    676
>    677	                                 0 1 2 3 4 5 6 7
>    678	                                +-+-+-+-+-+-+-+-+
>    679	                                |E|R|  Value    |
>    680	                                +-+-+-+-+-+-+-+-+
>    681
>    682	                       Figure 6: PDR-ACK status Format
>    683
>    684	      E:  1-bit flag.  Set to indicate a rejection.  When not
> set, the
>    685	         value of 0 indicates Success/Unqualified acceptance
> and other
>    686	         values indicate "not an outright rejection".
>    687	      R:  1-bit flag.  Reserved, MUST be set to 0 by the sender
> and
>    688	         ignored by the receiver.
>    689	      Status Value:  6-bit unsigned integer.  Values depending
> on the
>    690	         setting of the 'E' flag, see Table 27 and Table 28.
>    691
>    692	   Reserved:  The Reserved field MUST initialized to zero by
> the sender
>                                              ^ be
>    693	      and MUST be ignored by the receiver
>    694
>    695	6.3.  Via Information Options
>    696
>    697	   A VIO signals the ordered list of IPv6 Via Addresses that
> constitutes
>    698	   the hops of either a Serial Track or a Segment of a more
> Complex
>    699	   Track.  A VIO MUST contain at least one Via Address, and a
> Via
>    700	   Address MUST NOT be present more than once, otherwise the
> VIO MUST be
>    701	   ignored.
> 
> If i understand it correctly, for the SR-VIO, the sequence of addresses
> would
> ultimately end up in the RPL steering header of packets. I browsed through
> RFC8754 and think i can not find a similar exclusion. So a justification
> why this optimization is done for SF-VIO would be good if you really want
> to keep it.

If you find the same address twice you made a loop. There is already logic to avoid that in several protocols.
Now the rule to ignore the VIO requires additional validation code that the LLN node is node necessarily ready to pay for.
Suggested text:

                                                       The VIO contains
      one or more addresses listed in the order of progression of the
      packet from Ingress to Egress.  In a No-Path Non-Storing Mode
      P-DAO, the list MUST be elided from the NSM-VIO since the state in
      the Ingress is erased regardless.  In all other cases, a VIO MUST
      contain at least one Via Address, and a Via Address MUST NOT be
      present more than once, which would create a loop.

      A node that processes a VIO MAY verify whether one of these
      conditions happen, and when so, it MUST ignore the P-DAO and
      reject it with a RPL Rejection Status of "Error in VIO" in the
      DAO-ACK, see Section 10.14.

> 
> For SF-VIO, this seems to be taken apart and every node listed creates a
> next-hop route. So if the same node is addressed twice, it could become
> confused, which instance of its own address to choose to install the route,
> right ?
> 

Right


> 
> And what does "ignore" mean anyhow, e.g.: would the Target Options still be
> acted
> upon ? It rather sounds to me as if you would want the whole Projected DAO
> message
> to be rejected ?

There's nothing else to act upon. The Targets are the destinations of the routes via the VIO.
Still I incorporated the clarification in the text above.


> 
> Is duplicate address the only case of a "broken" projected DAO that you can
> discover and reject ? E.g.: can nodes have multiple addresses ? Such as
> anycast addresses ?
> 

Added that the addresses must be ULA or GUA. Other checks are listed in the new text 


> Would be good to elaborate about such "broken" projected DAO more if there
> are different
> cases of it.

Did some but note that an LLN node will probably NOT od the checks, due to constrained code space.
The MUST really applies to the Root that forms the VIO.

> 
>    701	   The format of the Via Information Options is as follows:
>    732
>    733	        0                   1                   2
> 3
>    734	        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
> 8 9 0 1
>    735	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    736	       |   Type        | Option Length |     Flags     |
> SegmentID   |
>    737	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    738	       |Segm. Sequence | Seg. Lifetime |      SRH-6LoRH header
> |
>    739	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    740	       |
> |
>    741	       +
> +
>    742	       .
> .
>    743	       .                     Via Address 1
> .
>    744	       .
> .
>    745	       +
> +
>    746	       |
> |
>    747	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    748	       |
> |
>    749	       .                              ....
> .
>    752	       |
> |
>    751	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    752	       |
> |
>    753	       +
> +
>    754	       .
> .
>    755	       .                     Via Address n
> .
> 
> add (n <= 15) ?

Why 15? Though this seems very large, SR-6LoRH may compress that to 30 bytes.
RFC 6554 does not provide an explicit limit.

> 
>    756	       .
> .
>    757	       +
> +
>    758	       |
> |
>    759	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    760
>    761
>    762	                  Figure 7: VIO format (uncompressed form)
>    763
>    764	   Option Type:  0x0B for SF-VIO, 0x0C for SR-VIO (to be
> confirmed by
>    765	      IANA)
> 
> Same concern as prior illegal occupations of IANA codepoints in this spec
> ;-))

Same answer I guess; note that RPL AODV will ship prior to this, so I'm pushing the suggested values.

> 
>    767	   Option Length:  In bytes; variable, depending on the number
> of Via
>    768	      Addresses and the compression.
> 
> Something like total size of the Option ... in bytes ?

Done 

> Is this semantic mandated by prior RFC ?  

RFC 6550, note that the first 2 bytes are not counted

> If it could count differently,
> it could allow for more than 15 hops, although i have no idea if that would
> be relevant.

Why 15? 

> 
>    770	   SegmentID:  8-bit field that identifies a Segment within a
> Track or
>    771	      the main DODAG as indicated by the TrackID field.  The
> value of 0
>    772	      is used to signal a Serial Track, i.e., made of a single
> segment.
> 
> Another basic signaling element not yet explained, so somewhat mysterious
> here

Reordered as you suggested
> 
>    774	   Segment Sequence:  8-bit unsigned integer.  The Segment
> Sequence
>    775	      obeys the operation in section 7.2 of [RPL] and the
> lollipop
>    776	      starts at 255.
>    777
>    778	      When the Root of the DODAG needs to refresh or update a
> Segment in
>    779	      a Track, it increments the Segment Sequence individually
> for that
>    780	      Segment.
>    781
>    789	      The Segment information indicated in the VIO deprecates
> any state
>    790	      for the Segment indicated by the SegmentID within the
> indicated
>    791	      Track and sets up the new information.
>    792
>    793	      A VIO with a Segment Sequence that is not as fresh as the
> current
>    794	      one is ignored.
>    795
>    796	      A VIO for a given DODAGID with the same (TrackID,
> SegmentID,
>    797	      Segment Sequence) indicates a retry; it MUST NOT change
> the
>    798	      Segment and MUST be propagated or answered as the first
> copy.
>    799
>    800	   Segment Lifetime:  8-bit unsigned integer.  The length of
> time in
>    801	      Lifetime Units (obtained from the Configuration option)
> that the
>    802	      Segment is usable.
>    803
>    804	      The period starts when a new Segment Sequence is seen.
> The value
>    805	      of 255 (0xFF) represents infinity.  The value of zero
> (0x00)
>    806	      indicates a loss of reachability.
>    807
>    808	      A P-DAO message that contains a VIO with a Segment
> Lifetime of
>    809	      zero is referred as a No-Path P-DAO in this document.
>    810
>    811	   SRH-6LoRH header:  The first 2 bytes of the (first) SRH-
> 6LoRH as
>    812	      shown in Figure 6 of [RFC8138].  A 6LoRH Type of 4 means
> that the
>    813	      VIA Addresses are provided in full with no compression.
>    814
>    815	   Via Address:  An IPv6 address along the Segment.
>    816
>    817	      In a SF-VIO, the list is a strict path between direct
> neighbors,
>    818	      from the Segment Ingress to Egress, both included.  The
> router
>    819	      that processes an SF-VIO MAY create additional routing
> state
>    820	      towards the nodes after self via the node immediately
> after self
>    821	      in the SF-VIO, but in case of memory shortage the routes
> to the
>    822	      Targets have precedence since they are the ones that the
> router
>    823	      commits to store.
> 
> Again, i am at a lack of understanding the base theory of operations of
> forwarding,
> I have Track Ingres, Track Egres, Destinations, Target and somehow i have
> Segments with addresses, wheree i do not know how Segment Ingres and
> Segment Egres
> may or may not map to Track Ingres/Egres, Destinations or Targets.
> 
> Picture me this, please ? ;-))

Done! It now looks like this and is early in the doc, moved to new section 3 after the terminology as opposed to within the terminology.



           CPF               CPF          CPF                 CPF

                          Southbound API

      _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-
    _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-

                         +----------+
                         | RPL Root |
                         +----------+
                           (      )
                 (                                  )
           (              DODAG                              )
             (                                           )
     (                                                         )
                                                                     )
     <-    Leg 1            B,                            E ->
     <--- Segment 1 A,B ---> <------- Segment 2 C,D,E ------->

                FWD  --z  Relay --z   FWD  --z   FWD          Target 1
            z-- Node  z--  Node  z--  Node  z--  Node --z     /
         --z    (A)        (B) \      (C)        (D)  z--    /
   Track                        \                       Track
   Ingress                    Segment 5                 Egress - Tgt 2
     (I)                           \                     (E)
         --z                        \                 z--    \
          z-- FWD   --z  FWD  --z  Relay --z  FWD  --z        \
              Node   z-- Node  z-- Node   z-- Node            Target 3
              (F)        (G)       (H)        (J)

     <------ Segment 3 F,G,H ------> <---- Segment 4 J,E ---->
     <-      Leg 2                  H,                    E ->

     <--- Segment 1 A,B ---> <- S5-> <---- Segment 4 J,E ---->
     <-      Leg 3          B,      H,                    E ->
                                                                     )
      (
                 (                                        )

                       Figure 2: Segments and Tracks


> 
> So, what exactly does the routing state look like ? whats the lookup ?
> Something like (RPLInstanceID==TrackID, Destination) ? Or rather: What is
> the example scenario when a the above described additional routing state
> to the nodes after self would would be used ?

All in section 3 now

> 
>    825	      In an SR-VIO, the list includes the egress but not the
> ingress
>    826	      node.  It starts at the first hop and ends at a Track
> Egress.  In
> 
> Seems like a mayor difference in flexiblity of stateful vs. stateless
> operations than ?
> No segment layer with SR-VIO ?

The segments are strict and join the loose SRH hops. See the SRH hops as DODAG edges (signaled as NSM) and Segments as vertices (signaled as SM).



> 
>    827	      that case, the Track Egress MUST be considered as an
> implicit
>    828	      Target, so it MUST NOT be listed it in a RPL Target
> Option.  The
>    829	      list in an SR-VIO may be loose, provided that each listed
> node has
>    830	      a path to the next listed node, e.g., via a segment or
> another
>    831	      Track.
> 
> But you are not telling me you would want to create the freedom to have
> stateless
> operation rely on routing state created by SF-VIO where you said it might
> not exist in case of memory shortage which the stateless encapsulator would
> know nothing about, right ?

The root should install the segments first and then join them with a track.


> 
>    833	      In the case of a SF-VIO, or if [RFC8138] is not used in
> the data
>    834	      packets, then the Root MUST use only one SRH-6LoRH per
> Via
>    835	      Information Option, and the compression is the same for
> all the
>    836	      addresses, as shown in Figure 7.
> 
> I do not see any address compression in Figure 7, nor was compression
> mentioned
> so far.

It's there but non obvious. The SRH-6LoRH header field is the beginning of it.


>
> Do you mean figure 7 in section 5.1 of [RFC8138] ?


I'll make it more visible.


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Type        | Option Length |     Flags     |   SegmentID   |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |Segm. Sequence | Seg. Lifetime |      SRH-6LoRH header         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    .           Via Address 1 (compressed by RFC 8138)              .
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    .                              ....                             .
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    .           Via Address n (compressed by RFC 8138)              .
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


> 
>    845	      In case of an SR-VIO, and if [RFC8138] is in use in the
> main
>    846	      DODAG, then the Root SHOULD optimize the size of the SR-
> VIO;
> 
> Any example how ?
> 
>    846         more
>    847	      than one SRH-6LoRH may be present, e.g., if the
> compression level
>    848	      changes inside the Segment and different SRH-6LoRH Types
> are
>    849	      required.
> 
> That rather sounds as if those are cases where optimiation would no be
> possible ?

Changed to 

                                  the Root SHOULD optimize the size of the
              NSM-VIO; this means that more than one SRH-6LoRH may be present,
              if the compression level changes inside the Segment and using
              different SRH-6LoRH Types make the VIO shorter.


> 
>    849        The content of the SR-VIO starting at the first SRH-
>    850	      6LoRH header is thus verbatim the one that the Track
> Ingress
>    851	      places in the packet encapsulation to reach the Track
> Ingress.
> 
> Example with picture would help.

Added

6.7.  Encapsulation and Compression

   When using [RFC8138] in the main DODAG operated in Non-Storing Mode
   in a 6LoWPAN LLN, a typical packet that circulates in the main DODAG
   is formatted as shown in Figure 13, representing the case where :

   +-+ ... -+- ... -+- ... -+-+- ... +-+-+-+ ... +-+-+ ... -+ ... +-...
   |11110001|  SRH- | RPI-  | IP-in-IP | NH=1      |11110CPP| UDP | UDP
   | Page 1 | 6LoRH | 6LoRH |  6LoRH   |LOWPAN_IPHC| UDP    | hdr |Payld
   +-+ ... -+- ... -+- ... -+-+- ... +-+-+-+ ... +-+-+ ... -+ ... +-...
                                        <=        RFC 6282      =>
             <================ Inner packet ==================== = =

           Figure 13: A Packet as Forwarded along the Main DODAG

   Since there is no page switch between the encapsulated packet and the
   encapsulation, the first octet of the compressed packet that acts as
   page selector is actually removed at encapsulation, so the inner
   packet used in the descriptions below start with the SRH-6LoRH, and
   is verbatim the packet represented in Figure 13 from the second octet
   on.

   When encapsulating that inner packet to place it in the Track, the
   first header that the Ingress appends at the head of the inner packet
   is an IP-in-IP 6LoRH Header; in that header, the encapsulator
   address, which maps to the IPv6 source address in the uncompressed
   form, contains a GUA or ULA IPv6 address of the Ingress node that
   serves as DODAG ID for the Track, expressed in the compressed form
   and using the DODAGID of the main DODAG as compression reference.  If
   the address is compressed to 2 bytes, the resulting value for the
   Length field shown in Figure 14 is 3, meaning that the SRH-6LoRH as a
   whole is 5-octets long.

      0                   1                   2
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-     ...     -+
     |1|0|1| Length  | 6LoRH Type 6  |  Hop Limit    | Track DODAGID |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-     ...     -+

                    Figure 14: The IP-in-IP 6LoRH Header

   At the head of the resulting sequence of bytes, the track Ingress
   then adds the RPI that carries the TrackID as RPLinstanceID as a P-
   RPI-6LoRH Header, as illustrated in Figure 5, using the TrackID as
   RPLInstanceID.  Combined with the IP-in-IP 6LoRH Header, this allows
   to identify the Track without ambiguity.

   The SRH-6LoRH is then added at the head of the resulting sequence of
   bytes as a verbatim copy of the content of the SR-VIO that signaled
   the selected Track Leg.

        0                   1
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5  ..         ..        ..
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-    -+-    -+ ... +-    -+
       |1|0|0|  Size   |6LoRH Type 0..4| Hop1 | Hop2 |     | HopN |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-    -+-    -+ ... +-    -+
                                                 Where N = Size + 1

                      Figure 15: The SRH 6LoRH Header

   The resulting encapsulated packet looks as illustrated in Figure 16:

   +-+ ... -+-+-+- ... -+-+-+- ... -+-+-+-+-+- ... +-+-+-+-+-+-+- ...
   | Page 1 |  SRH-6LoRH  | P-RPI-6LoRH | IP-in-IP 6LoRH | Inner Packet
   +-+ ... -+-+-+- ... -+-+-+- ... -+-+-+-+-+- ... +-+-+-+-+-+-+- ...

    Signals :  Loose Hops :    TrackID  :  Track DODAGID :

           Figure 16: A Packet as Forwarded along a Track

> 
>    853	6.4.  Sibling Information Option
>    854
>    855	   The Sibling Information Option (SIO) provides indication on
> siblings
>    856	   that could be used by the Root to form Projected Routes.
> One or more
>    857	   SIO(s) may be placed in the DAO messages that are sent to
> the Root in
>    858	   Non-Storing Mode.
> 
> Is this some poor-mans (little signaling) form of signaling to help the
> root do some topology discovery ? If so, then it probably would be
> something
> that needs to go to the PCE, right ? Maybe this type of high level
> explanation in the tex would be helpful. If i am guessing wrong then i even
> understand
> less.

You're correct

Added

    This specification expects that the main RPL Instance is operated in RPL
    Non-Storing Mode to sustain the exchanges with the Root. Based on its
    comprehensive knowledge of the parent-child relationship, the Root can form
    an abstracted view of the whole DODAG topology. This document adds the
    capability for nodes to advertise additional sibling information to
    complement the topological awareness of the Root to be passed on to the PCE.


> 
>    859
>    860	   The format of the SIO is as follows:
>    861
>    862	        0                   1                   2
> 3
>    863	        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
> 8 9 0 1
>    864	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    865	       |   Type        | Option Length |Comp.|B|D|Flags|
> Opaque     |
>    866	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    867	       |            Step of Rank       |          Reserved
> |
>    868	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    869	       |
> |
>    870	       +
> +
>    871	       .
> .
>    872	       .       Sibling DODAGID (if the D flag not set)
> .
>    873	       .
> .
>    874	       +
> +
>    875	       |
> |
>    876	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    877	       |
> |
>    878	       +
> +
>    879	       .
> .
>    880	       .                     Sibling Address
> .
>    881	       .
> .
>    882	       +
> +
>    883	       |
> |
>    884	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> +-+-+-+-+
>    885
>    886	                Figure 8: Sibling Information Option Format
>    887
>    888	   Option Type:  0x0D (to be confirmed by IANA)
> 
> You're maxing out your repeat offender penalty of illegal IANA code point
> misappropriation ;-)

Not illegal, just suggested, added (to be confirmed by IANA)

> 
>    890	   Option Length:  In bytes, the size of the option.
>    891
>    892	   Compression Type:  3-bit unsigned integer.  This is the SRH-
> 6LoRH
>    901	      Type as defined in figure 7 in section 5.1 of [RFC8138]
> that
>    902	      corresponds to the compression used for the Sibling
> Address and
>    903	      its DODAGID if  resent.  The Compression reference is the
> Root of
>                              ^ p ?
>    904	      the main DODAG.
>    905
>    906	   Reserved for Flags:  MUST be set to zero by the sender and
> MUST be
>    907	      ignored by the receiver.
>    908
>    909	   B:  1-bit flag that is set to indicate that the connectivity
> to the
>    910	      sibling is bidirectional and roughly symmetrical.  In
> that case,
>    911	      only one of the siblings may report the SIO for the hop.
> If 'B'
> 
> How would they do that (decide which one reports the SIO ?).
> 
>    911	      only one of the siblings may report the SIO for the hop.
> If 'B'
>    912	      is not set then the SIO only indicates connectivity from
> the
>    913	      sibling to this node, and does not provide information on
> the hop
>    914	      from this node to the sibling.
> 
> I hope there is a simple way to do 'B' or more of a benefit than just
> cutting
> down reporting data by 50%. Any standardized mechanism ?*


OK, I remove that B flag for now, that's optimization. In the P-DAO discussion I added

                                                          If this router also
   registers an address to that sibling, and the link has similar properties in
   both directions, only the router with the lowest Interface ID in its
   registered address needs report the SIO, and the Root will assume symmetry.


> 
>    916	   D:  1-bit flag that is set to indicate that sibling belongs
> to the
>    917	      same DODAG.  When not set, the Sibling DODAGID is
> indicated.
>    918
>    919	   Flags:  Reserved.  The Flags field MUST initialized to zero
> by the
>                                                   ^ be
>    920	      sender and MUST be ignored by the receiver
>    921
>    922	   Opaque:  MAY be used to carry information that the node and
> the Root
>    923	      understand, e.g., a particular representation of the Link
>    924	      properties such as a proprietary Link Quality Information
> for
>    925	      packets received from the sibling.  An industrial
> Alliance that
>    926	      uses RPL for a particular use / environment MAY redefine
> the use
>    927	      of this field to fit its needs.
> 
> Is there a prior RPL example of such a field ? Pointer to that wold be
> useful. I hav a hard time seeing how this would not result in all type of
> misinterpretation unless the industrial alliance whose registration space
> is to be used for Upaque is unambigously derivable from the device (type)
> or link (type).
> 
> I would prefer to add MUST be set to 0 and be ignored upon receiving and
> go on saying, IETF specification MUST NOT use or define this field, but
> non-IETF specification may use it. Aka: why not give such external specs
> the same starting ground (0-filled) that we give our own extension points.
> 
> But again: pre-existing examples of BCP spec text for such an Opaque field
> would
> be helpful.


See RFC 8505 fig 1. 
The reason for specifying opaque is to ensure that it can be used by another party within their std that uses RPL, and there will not be an RFC later that reuses the field and creates a collision.



> 
>    929	   Step of Rank:  16-bit unsigned integer.  This is the Step of
> Rank
>    930	      [RPL] as computed by the Objective Function between this
> node and
>    931	      the sibling.
>    932
>    933	   Reserved:  The Reserved field MUST initialized to zero by
> the sender
>    934	      and MUST be ignored by the receiver
>    935
>    936	   Sibling DODAGID:  2 to 16 bytes, the DODAGID of the sibling
> in a
>    937	      [RFC8138] compressed form as indicated by the Compression
> Type
>    938	      field.  This field is present if and only if the D flag
> is not
>    939	      set.
>    940
>    941	   Sibling Address:  2 to 16 bytes, an IPv6 Address of the
> sibling, with
>    942	      a scope that MUST be make it reachable from the Root,
> e.g., it
>    943	      cannot be a Link Local Address.  The IPv6 address is
> encoded in
>    944	      the [RFC8138] compressed form indicated by the
> Compression Type
>    945	      field.
>    956
>    957	   An SIO MAY be immediately followed by a DAG Metric
> Container.  In
>             ^ "A for consonant start of TLA"
>    958	   that case the DAG Metric Container provides additional
> metrics for
>    959	   the hop from the Sibling to this node.
>    960
>    961	7.  Projected DAO
> 
> Given how this section seems to describe what the title of the doc is,
> maybe it should also be named that way "Root initiated routing state".

I like that... Done

> 
>    962
>    963	   This draft adds a capability to RPL whereby the Root of a
> main DODAG
> 
> Define what a "main DODAG" is. I guess its a DODAG not created by the
> mechanisms of this draft, but also: what are non-main DODAGs ? Or is the
> opposite just the projected DAO ?

Added in intro
"
   an external controller interacts with the RPL Root to compute centrally
   shorter Peer to Peer (P2P) paths within a pre-existing RPL Main DODAG.
"

Actually the system may recurse. A Track is a DODAG that may becomes a Main DODAG for tracks within tracks. But that takes us too far for this document.


> 
>    964	   installs a Track as a collection of Projected Routes, using
> a
>    965	   Projected-DAO (P-DAO) message to maintain each individual
> route.  The
> 
> What is an individual route ?

Changed to 

"
  for each individual Track Leg or Segment.
"
> 
>    966	   P-DAO signals a collection of Targets in the RPL Target
> Option(s)
>    967	   (RTO).  Those Targets can be reached via a sequence of
> routers
>    968	   indicated in a VIO.  A P-DAO message MUST contain exactly
> one VIO,
>    969	   which is either a SF-VIO or an SR-VIO, and MUST follow one
> or more
>    970	   RTOs.  There can be at most one such sequence of RTO(s) and
> an Via
>    971	   Information Option.  A track is identified by a tuple
> DODAGID,
>                              ^ in a P-DAO
> 
> Also, is "at most" correct ?
> How about there MUST be exactly one sequece of one or more RTO followed vy
> one VIO ?
> Or is it valid to have no RTO, or no VIO ?

I like your wording, simpler and to the point. 

> 
>    972	   TrackID and each route within a Track is indexed by a
> SegmentID.
> 
> So, IMHO, this whole section 7 needs to come before section 3, because it
> is
> here where you start defining the functionality from higher layer, but then
> section 3 goes into the encoding details. I observe that all RFCs i
> remember
> have the order of starting wih the overall functionality and bring the
> encoding
> details of messages later (but then i am forgetfull of what i have read and
> also say this because as my comments before will show, i struggled  to make
> head & tails out of how the pieces fit together).
> 

I did reshuffle stuff quite a bit. Now it looks like
   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
     2.1.  Requirements Language . . . . . . . . . . . . . . . . . .   4
     2.2.  References  . . . . . . . . . . . . . . . . . . . . . . .   4
     2.3.  Glossary  . . . . . . . . . . . . . . . . . . . . . . . .   4
     2.4.  Domain Terms  . . . . . . . . . . . . . . . . . . . . . .   5
   3.  Context and Goal  . . . . . . . . . . . . . . . . . . . . . .   6
     3.1.  RPL Applicability . . . . . . . . . . . . . . . . . . . .   6
     3.2.  RPL Routing Modes . . . . . . . . . . . . . . . . . . . .   7
     3.3.  On Tracks . . . . . . . . . . . . . . . . . . . . . . . .   8
     3.4.  Serial Track Signaling  . . . . . . . . . . . . . . . . .   9
       3.4.1.  Using Storing-Mode Segments . . . . . . . . . . . . .  10
       3.4.2.  Using Non-Storing-Mode joining Tracks . . . . . . . .  15
     3.5.  Complex Tracks  . . . . . . . . . . . . . . . . . . . . .  22
     3.6.  Scope and Expectations  . . . . . . . . . . . . . . . . .  24
   4.  Extending existing RFCs . . . . . . . . . . . . . . . . . . .  26
     4.1.  Extending RFC 6550  . . . . . . . . . . . . . . . . . . .  26
       4.1.1.  Projected DAO . . . . . . . . . . . . . . . . . . . .  26
       4.1.2.  Via Information Option  . . . . . . . . . . . . . . .  28
       4.1.3.  Sibling Information Option  . . . . . . . . . . . . .  28
       4.1.4.  P-DAO Request . . . . . . . . . . . . . . . . . . . .  28
       4.1.5.  Extending the RPI . . . . . . . . . . . . . . . . . .  29
     4.2.  Extending RFC 6553  . . . . . . . . . . . . . . . . . . .  29
     4.3.  Extending RFC 8138  . . . . . . . . . . . . . . . . . . .  30
   5.  New RPL Control Messages and Options  . . . . . . . . . . . .  31
  


>    973
>    974	   A P-DAO MUST be sent from the address of the Root that
> serves as
>    975	   DODAGID for the main DODAG.  It MUST be sent to a GUA or a
> ULA of
> 
> Expand GUA, ULA, provide reference for ULA, and/or add to initial list of
> terms in doc.

Added in first use and in acronyms

> 
>    976	   either the ingress or the egress of the Segment, more below.
> If the
>    977	   'K' Flag is present in the P-DAO, and unless the P-DAO does
> not reach
>    978	   it, the ingress of the Segment is the node that acknowledges
> the
>    979	   message, using a DAO-ACK that MUST be sent back to the
> address that
>    980	   serves as DODAGID for the main DODAG.
> 
> Sentence too convoluted. Try to make two out of it.

What about


    If the 'K' Flag is present in the P-DAO, the P-DAO must be acknowledged
    using a DAO-ACK that is sent back to the address of the Root from which the
    P-DAO was received. In most cases, the first node of the Leg, Segment,
    or updated section of the Segment is the node that sends the acknowledgment.
    The exception to the rule is when an intermediate node in a Segment fails
    to forward a Storing Mode P-DAO to the previous node in the SM-VIO; in that
    case, the node that fails to forward the P-DAO sends the acknowledgment.
    The DAO-ACK MUST be sent back to the address that serves as DODAGID for the
    Main DODAG.
> 
> You are not using the same term here as in 974, so this makes one wonder if
> the DAO-ACK is to be sent back to the same address it was sent from or not.
> If it is meant to be sent back to the same address, why not say "sent back
> to
> the address of the root, which the P-DAO was sent from".

Adopted in the text above


> 
>    982	   Like a classical DAO message, a P-DAO causes a change of
> state only
>    983	   if it is "new" per section 9.2.2.  "Generation of DAO
> Messages" of
>    984	   the RPL specification [RPL]; this is determined using the
> Segment
>    985	   Sequence information from the VIO as opposed to the Path
> Sequence
>                                             ^ in the P-DAO
>    986	   from a TIO.  Also, a Segment Lifetime of 0 in a VIO
> indicates that
>                      ^ in the DAO.
>    987	   the projected route associated to the Segment is to be
> removed.
>    988
>    989	   There are two kinds of operation for the Projected Routes,
> the
>    990	   Storing Mode and the Non-Storing Mode.
>    991
>    992	   *  The Non-Storing Mode is discussed in Section 7.3.2.  A
> Non-Storing
>    993	      Mode P-DAO carries an SR-VIO with the loose list of Via
> Addresses
>                                                          ^ steering
>    994	      that forms a source-routed Segment to the Track Egress.
> The
>    995	      recipient of the P-DAO is the Track Ingress; it MUST
> install a
>    996	      source-routed state to the Track Egress and reply to the
> Root
>    997	      directly using a DAO-ACK message if requested to.
>    998
>    999	   *  The Storing Mode is discussed in Section 7.3.1.  A
> Storing Mode
>   1000	      P-DAO carries a  SF-VIO with the strict  list of Via
> Addresses from
>                              ^n                      ^ steering
>   1001	      the ingress to the egress of the Segment in the data path
> order.
>   1002	      The routers listed in the Via Addresses, except the
> egress, MUST
>   1003	      install a routing state to the Target(s) via the next Via
> Address
>   1004	      in the SF-VIO.  In normal operations, the P-DAO is
> propagated
>   1013	      along the chain of Via Routers from the egress router of
> the path
>   1014	      till the ingress one, which confirms the installation to
> the Root
>   1015	      with a DAO-ACK message.
> 
> In 976 you point to further below as to where the P-DAO is to be sent. But
> hen
> in the bullet points 992 and 999 you do not explicitly say tha the SR-VIO
> is sent
> to the track ingres and that the SF-VIO is sent to the track egres. Please
> add such sentences accordingly.
> 
added

    A P-DAO that creates or updates a Track Leg MUST be sent to a GUA or a ULA
    of the Ingress of the Leg; it must contain the full list of hops in the
    Leg unless the Leg is being removed.
    A P-DAO that creates a new Track Segment MUST be sent to a GUA or a ULA
    of the Segment Egress and signal the full list of hops in Segment; a
    P-DAO that updates (including deletes) a section of a Segment MUST be
    sent to the first node after the modified Segment and signal the full
    list of hops in the section starting at the node that immediately
    precedes the modified section.




> I think the description of ACK processing from 976++ fits better here
> after, given
> how its an additional detail happening after the P-DAO is processed
> according
> to the two bullet points above.
> 
> Q: If in the SF-VIO case, the egress does not install any state, then why
> send
> the P-DAO to it instead of to the pre-ultimate hop ? Explanation might be
> helpful
> in the text.

The P-DAO goes along the reverse path that the packets will follow and check the path on the way.

> 
>   1016
>   1017	   In case of a forwarding error along a Projected Route, an
> ICMP error
> 
> Arre we talking about the forwaring of data packets that are following a
> projected route,
> or processing error of the P-DAO ? Please be specific. An example of a
> forwarding
> error would help.



   In case of a failure forwarding a packet along a P-Route, e.g., the
   next hop is unreachable, the node that discovers the fault MUST send
   an ICMPv6 Error message [RFC4443] to the Root, with a new Code "Error
   in P-Route" (See Section 10.13). The message MUST be throttled in
   case of consecutive occurrences.


> I suspect it is the data packets. Is there also any other error for
> processing of
> the P-DAO other than missing ACK ? Aka: wha exactly happens in the
> duplicate address
> in VIO If lets say the root was too stsupid to figure it out ?



Per your earlier point I added

   A node that processes a VIO MAY verify whether one of these
   conditions happen, and when so, it MUST ignore the P-DAO and reject
   it with a RPL Rejection Status of "Error in VIO" in the DAO-ACK, see
   Section 10.14.

> 
>   1018	   is sent to the Root with a new Code "Error in Projected
> Route" (See
>   1019	   Section 11.13).  The Root can then modify or remove the
> Projected
>   1020	   Route.  The "Error in Projected Route" message has the same
> format as
>   1021	   the "Destination Unreachable Message", as specified in RFC
> 4443
>   1022	   [RFC4443].
>   1023
>   1024	   The portion of the invoking packet that is sent back in the
> ICMP
>   1025	   message SHOULD record at least up to the RH if one is
> present, and
>   1026	   this hop of the RH SHOULD be consumed by this node so that
> the
>            ^^^^                                     ^^^^
> which hop ? The hop that has problems and wants to generate the ICMP error?

Clarified as:
   
   In case of a failure forwarding a packet along a P-Route, e.g., the
   next hop is unreachable, the node that discovers the fault MUST send
   an ICMPv6 Error message [RFC4443] to the Root, with a new Code "Error
   in P-Route" (See Section 10.13).  The Root can then repair by
   updating the broken Segment and/or Tracks, and in the case of a
   broken Segment, remove the leftover sections of the segment using SM-
   VIOs with a lifetime of 0 indicating the section ot one or more nodes
   being removed (See Section 6.5).
> 
>   1027	   destination in the IPv6 header is the next hop that this
> node could
>   1028	   not reach.  if a 6LoWPAN Routing Header (6LoRH) [RFC8138] is
> used to
> 
> So ultimately, you want the root to be able to figure out from the ICMP,
> which nodeA can not reach a next-hop nodeB, right ? WOuld be good to say
> that,
> but the description of how to achieve this by messing with the ICMP error
> reported original packet stub still escapes me. Is that "consumed by"
> procedure
> already done some place else ? If so, hen a reference to prior way how this
> is
> done would be good. If not, then elaborating a bit more to make it easier
> to
> understand would help.

I added 

   In case of a failure forwarding a packet along a Segment, e.g., the
   next hop is unreachable, the node that discovers the fault MUST send
   an ICMPv6 Error message [RFC4443] to the Root, with a new Code "Error
   in P-Route" (See Section 10.13).  The Root can then repair by
   updating the broken Segment and/or Tracks, and in the case of a
   broken Segment, remove the leftover sections of the segment using SM-
   VIOs with a lifetime of 0 indicating the section ot one or more nodes
   being removed (See Section 6.5).

   In case of a forwarding error along a Source Route path, the node
   that fails to forward SHOULD send an ICMP error with a code "Error in
   Source Routing Header" back to the source of the packet, as described
   in section 11.2.2.3. of [RPL].  Upon this message, the encapsulating
   node SHOULD stop using the source route path for a period of time and
   it SHOULD send an ICMP message with a Code "Error in P-Route" to the
   Root.  Failure to follow these steps may result in packet loss and
   wasted resources along the source route path that is broken.

   Either way, the ICMP message MUST be throttled in case of consecutive
   occurrences.  It MUST be sourced at the ULA or a GUA that is used in
   this Track for the source node, so the Root can establish where the
   error happened.

> 
>   1028	   not reach.  if a 6LoWPAN Routing Header (6LoRH) [RFC8138] is
> used to
>   1029	   carry the IPv6 routing information in the outer header then
> that
>   1030	   whole 6LoRH information SHOULD be present in the ICMP
> message.
> 
> How would a root then figure out nodeA/nodeB ?

Added text above

> 
>   1031
>   1032	   The sender and exact operation depend on the Mode and is
> described in
>   1033	   Section 7.3.2 and Section 7.3.1 respectively.
>   1034
>   1035	7.1.  Requesting a Track
>   1036
>   1037	   A Node is free to ask the Root for a new Track for which it
> will be
>   1038	   Ingress at any time.  This is done with a PDR message, that
> indicates
> 
> for which it wants to be ingres ? E.g.: no idea what "will be at any time"
> means.

added

      This specification introduces the PDR message, used by an LLN node to
      request the formation of a new Track for which this node is Ingress.

> 
>   1039	   the desired TrackID and the duration for which the Track
> should be
>   1040	   established.  Upon a PDR, the Root MAY install the necessary
> 
> An example of why the node would generate it, and where it has the TrackID
> from
> would help here.
> 
>   1041	   Segments, in which case it answers with a PDR-ACK indicating
> the
>   1042	   granted Track Lifetime.  All the Segments MUST be of a same
> mode,
>   1043	   either Storing or Non-Storing.  All the Segments MUST be
> created with
>   1044	   the same TrackID and the same DODAGID signaled in the P-DAO.
>   1045
>   1046	   The Root is free to design the Track as it wishes, and to
> change the
>   1047	   Segments overtime to serve the Track as needed, without
> notifying the
>   1048	   resquesting Node.  The Segment Lifetime in the P-DAO
> messages does
>              ^
> Presumably, a requesting node would likely also be the ingres of one P-DAO
> of
> the track, right ? But establishment of the track via P-DAO is only related
> to
> the PDR/PDR-ACK in the root ?

This is not specified. The Root could also decide to form a track based on application request, but then there must be a method to avoid collision in the trackID name space.



Added

      This specification introduces the PDR message, used by an LLN node to
      request the formation of a new Track for which this node is Ingress.
      Note that the namespace for the TrackID is owned by the Ingress node,
      and in the absence of a PDR, there must be some procedure for the Root to
      assign TrackIDs in that namespace while avoiding collisions.


> Might be useful to have some place a higher level description of
> "initiation" models,
> and point to it. Aka: the way i figure, initiation can come from PCE and go
> through root,
> no other nodes involved. Or it comes somehow via PDR initiaor... but
> how/why.
> 
> Also "free to " is a bit of a misnomer, right ? Aka: If the root is free,
> then it

Removed free

> should also be able to notify the requesting node, but there really is no
> other
> notification than acknowledging in PDR-ACK acceptance to take care of the
> request
> or to indicate failure of it. Aka, write maybe: "The root designs the Track
> as it wishes....
> there are no notifications to the requesting node when changing the track".

Done
	
> 
>   1049	   not need to be aligned to the Requested Lifetime in the PDR,
> or
>   1050	   between P-DAO messages for different Segments.  The Root may
> use
>   1051	   shorter lifetimes for the Segments and renew them faster
> than the
>   1052	   Track is, or longer lifetimes in which case it will need to
> tear down
>   1053	   the Segments if the Track is not renewed.
>   1054
>   1055	   When the Track Lifetime that was returned in the PDR-ACK is
> close to
>   1056	   elapse, the resquesting Node needs to resend a PDR using the
> TrackID
>   1057	   in the PDR-ACK to extend the lifetime of the Track, else the
> Track
>   1058	   will time out and the Root will tear down the whole
> structure.
> 
> How close ? Any guidance ?

Depends on the LLN
Added 

          - vs. the trip time from the node to the Root
> 
>   1068
>   1069	   If the Track fails and cannot be restored, the Root notifies
> the
>   1070	   resquesting Node asynchronously with a PDR-ACK with a Track
> Lifetime
>              ^
>   1071	   of 0, indicating that the Track has failed, and a PDR-ACK
> Status
>   1072	   indicating the reason of the fault.
>   1073
>   1074	7.2.  Identifying a Track
>   1075
>   1076	   RPL defines the concept of an Instance to signal an
> individual
>   1077	   routing topology but does not have a concept of an
> administrative
>   1078	   distance, which exists in certain proprietary
> implementations to sort
>   1079	   out conflicts between multiple sources of routing
> information within
>   1080	   one routing topology.
>   1081
>   1082	   This draft leverages the RPL Instance model as follows:
> 
> ... to sort out conflicts between multiple sources of routing information
> ??
> aka: tie the text together, the way it is written it is confusing.
> 
> It also seems as if it would be better to high level answer how P-DAO
> avoids
> the need for administrative distances here, instead of letting readers try
> to figure it
> out from all the details of the following bullet points, because those
> bullet points
> explain a lot more, so the whole administative distance stuff will get
> lost.

Not critical for understanding the section
Changed to

     RPL defines the concept of an Instance to signal an individual
     routing topology, and multiple topologies can coexist in the same network.
     The RPLInstanceID is tagged in the RPI of every packet to signal which
     topology the packet actually follows.

> 
> The way i see it is that P-DAO do not require administrative distance
> because
> 
> a) They will be preferred because of prefixlength vs. default route to root
> when
>    used within pre-existing RPLinstance.
> b) They can be explicitly indicated through their own RPLinstance called
> TrackID
>    in the RPL packet header, therefore not colliding with routes from pre-
> existing
>    RPLinstances.

Ack; the problem will arise when 2 tracks get to the same destination, which we suggest to avoid. And in storing mode since more specific route exist in the dodag which may conflict.

>   1083
>   1084	   *  The Root MAY use P-DAO messages to add better routes in
> the main
>   1085	      (Global) Instance in conformance with the routing
> objectives in
>   1086	      that Instance.  To achieve this, the Root MAY install an
> Storing-
> 
> What actually happens if those P-DAO routes are not in conformance with the
> routing
> objectives of that instance ? Anything worse than that the routing
> objective does
> not well express what the routes are doing ?

Could write a book. Same as SDN...


> 
> I am asking, because i would assume that the routing objectives likely will
> never
> be able to express all the characteristics that may come out of a bunch of
> Tracks.
> Therefore i fear that this conformance sentence is somewhat career limiting
> to
> P-DAOs.
> 
Yes, a real complex problem. How do you map appl level SLA to network concepts?


> But i am not sure about the overall framework. Is there some form of policy
> framework,
> whereby traffic may choose one out of multiple instances, and what you
> really want
> to say is that Tracks should be assigned to the track that best matches
> what the
> Track attempts to improve upon so that traffic choosing one out of multiple
> insances can continue to do so, and track will improve the desired outcome
> ?
> 

All that policy game is out of scope but has to be considered soon after.
For now we build a shorted path for a more specific route, without characterization of how good it is.


> Aka: Motivating this setnence, maybe in the way i am imagining it (if
> correct) might
> help. And it might also better capture the goal of choosing a particular
> instance
> for a track than referring to the formal routing objectives instead of
> maybe
> the informal intent of an instance, which may be constituted from the
> routing
> objectives of DAOs and the traffic engineered objectives of P-DAOs.


Out of scope, keep for next

> 
>   1086	      that Instance.  To achieve this, the Root MAY install an
> Storing-
>                                                                      ^ just
> a - storing is spoken with consonant first.
>   1087	      Mode P-Route along a path down the main Non-Storing Mode
> DODAG.
>   1088	      This enables a loose source routing and reduces the size
> of the
>   1089	      Routing Header, see Appendix A.1.
>   1090
>   1091	      When adding an Storing-Mode P-Route to the main RPL
> Instance, the
>                            ^ just a
>   1092	      Root MUST set the RPLInstanceID field of the P-DAO
> message (see
>   1093	      section 6.4.1. of [RPL]) to the RPLInstanceID of the main
> DODAG,
>   1094	      and MUST NOT use the DODAGID field.  A Projected Route
> provides a
> 
> Whats the significance of the DODAGID here ? What would happen if it was
> set ?

a) see RPL and all the discussion on tuple identification of Track. Adding it would waste bandwidth. That is consistent with RPL.

> 
>   1095	      longer match to the Target Address than the default route
> via the
>   1096	      Root, so it is preferred.
>   1097
>   1098	      Once the Projected Route is installed, the intermediate
> nodes
>   1099	      listed in the SF-VIO after first one (i.e.  The ingress)
> can be
>   1100	      elided from the RH in packets sent along the Segment
> signaled in
>   1101	      the P-DAO.  The resulting loose source routing header
> indicates
>   1102	      (one of) the Target(s) as the next entry after the
> ingress.
>   1103
>   1104	   *  The Root MAY also use P-DAO messages to install a
> specific (say,
>   1105	      Traffic Engineered) path as a Serial or as a Complex
> Track, to a
> 
> Serial Track and Complex Track are not defined/explained yet. See comments
> earlier that you need to start off with pictured examples from which to
> much more easily define such terms.
> 

Now it is, in great details, in section 3


>   1106	      particular endpoint that is the Track Egress.  In that
> case, the
>   1107	      Root MUST install a Local RPL Instance (see section 5 of
> [RPL]),
>   1108	      and the Local RPLInstanceID is called TrackID.
> 
> Do i need a separate RPL instance to loosely tie together multiple
> sequential
> SF-VIO ?  Lets say i have 20 hops in stateless. I figure i have 3 segments
> of
> 5 hops each that ae often used. So i establish separate tracks for each of
> them, and then my routing headers in the future would be able to be reduced
> by 3 * 5 = 15 hops, aka: having 3 loose hops tied together by 5 strict
> hops.

Sure, that's a main goal of this spec. see A.1.  Loose Source Routing. 

This is now clarified in section 3

  A P-Route may be installed in either Storing and Non-Storing Mode,
   potentially resulting in hybrid situations where the Mode of the P-
   Route is different from that of the main RPL Instance.  P-Routes can
   be used as stand-alone segments to reduce the size of the source
   routing headers with loose source routing operations down the main
   RPL DODAG.  P-Routes can also be combined with other P-Routes to form
   a more complex forwarding graph called a Track.


> 
>   1109
>   1110	      In that case, the TrackID MUST be unique for the Global
> Unique
>   1111	      IPv6 Address (GUA) or Unique-Local Address (ULA) of the
> Track
> 
> You where missing those expansions in before, so here they would now be
> redundant
> when you fix them up in before.

Resolved

> 
>   1112	      Ingress that serves as DODAGID for the Track.  The Track
> Ingress
>   1113	      owns the namespace of its TrackIDs, so it can pick any
> unused
>   1114	      value to request a new Track with a PDR.  The Root is
> aware of all
> 
> The root of the main DODAG, or main RPL instance or the like ? Aka: There
> may be
> multiple roots for different instances/dodags, right ?

Right. Changed to

     In that case, the TrackID MUST be unique for the IPv6 ULA or GUA of the
     Track Ingress that serves as DODAGID for the Track. The Track Ingress owns
     the namespace of its TrackIDs, so it can pick any unused value to request a
     new Track with a PDR. In a particular deployment where PDR are not used,
     the namespace can be delegated to the main Root, which can  assign the
     TrackIDs for the Tracks it creates without collision.


> 
>   1115	      the active Tracks, so it can also pick any unused value
> to form
>   1116	      Tracks without a PDR.  To avoid a collision of the Root
> and the
>   1125	      Track Ingress picking the same value at the same time, it
> is
>   1126	      RECOMMENDED that the Track Ingress starts allocating the
> ID value
>   1127	      of the Local RPLInstanceID (see section 5.1. of [RPL])
> used as
>   1128	      TrackIDs with the value 0 incrementing, while the Root
> starts with
>   1129	      63 decrementing.
> 
>   1131	      This way, a Track is uniquely identified by the tuple
> (DODAGID,
>   1132	      TrackID) where the TrackID is always represented with the
> D flag
>   1133	      set to 0.
> 
> Explain where the D flag is, first time i think its used in the text.
  
Added 

      This way, a Track is uniquely identified by the tuple (DODAGID,
      TrackID) where the TrackID is always represented with the D flag
      set to 0 (see also section 5.1. of [RPL]), indicating when used in
      an RPI that the source address of the IPv6 packet signals the
      DODAGID.

> 
> I think i may be confused about the namespace and its limitations or at
> least
> not well educaed by the text ;-)
> 
> So the track ingres IPv6 serves as DODAGID. So every node can have up to 64
> - N Tracks,
> where N is the number of non-rack RPL instances for which the node is the
> root ?

Yes


Added 

   Section 5.1. of [RPL] describes the RPL Instance ID and its encoding.
   There can be up to 128 global RPL Instances, for which there can be
   one or more DODAGs, and there can be 64 local RPL Instances, with a
   namespace that is indexed by a DODAGID, where the DODAGID is a Unique
   Local Address (ULA) or a Global Unicast Address (GUA) of the Root of
   the DODAG.


> 
> Would be good to write this out explicitly (or accordingly corrected if
> wrong).
> 
>   1135	      The Track Egress Address and the TrackID MUST be signaled
> in the
>   1136	      P-DAO message as shown in Figure 1.
>   1137
>   1138	7.3.  Installing a Track
>   1139
>   1140	   A Storing-Mode P-DAO contains an SF-VIO that signals the
> strict
>   1141	   sequence of consecutive nodes to form a segment between a
> segment
>   1142	   ingress and a segment egress (both included).  It installs a
> route of
>   1143	   a higher precedence along the segment towards the Targets
> indicated
>   1144	   in the Target Options.  The segment is included in a DODAG
> indicated
> 
> Ok, now i am getting confused again by this text.
> 
> "higher precendence - than what ? Also, is there a definition of precedence
> in the
> context of RPL ? If so, reference to definition would be good.

Added 

   A route along a Track for which the TrackID is not the RPLInstanceID
   of the Main DODAG MUST be installed with a higher precedence than the
   routes along the Main DODAG, meaning that:

   *  Longest match MUST be the prime comparison for routing.

   *  In case of equal length match, the route along the Track MUST be
      preferred vs. the one along the Main DODAG.

   *  There SHOULD NOT be 2 different Tracks leading to the same Target
      from same Ingress node, unless there's a policy for selecting
      which packets use which Track; such policy is out of scope.

   *  A packet that was routed along a Track MUST NOT be routed along
      the main DODAG again; if the destination is not reachable as a
      neighbor by the node where the packet exits the Track then the
      packet MUST be dropped.



> 
> Previously you only used precedence related to priority to store/maintain
> a route, which i guess is a differrent meaning of the word than here.
> 
> In 7.2, it sounded as if there would be no pre-existing route to the target
> other than through the root because of prefix-length. If that is what you
> mean
> here, and unless precedence has a well defined meaning, maybe just replace
> precedence
> with prefix-length.
> 
> It is also not quite clear to me if tracks can never be installed into
> storing-mode
> RPL instances where they could not conflict with equal-prefix-length
> "native" routes.
> AFAIK, the spec only said the main RPLinstance must be non-storing mode.
> Can there nit
> be other, storing mode RPL instances ?

You're spot on

Before your review and the possibility of a storing mode main dodag there could not be a prefix length collision. 
So the precedence was implicit due to longest match, though nothing was said about it.

I hope the above clarifies it all.



> 
> But of course, i am likely confused...

Certainly not

> 
>   1144	   in the Target Options.  The segment is included in a DODAG
> indicated
>   1145	   by the P-DAO Base Object, that may be the one formed by the
> main RPL
>   1146	   Instance, or a Track associated with a local RPL Instance.
> A Track
>   1147	   Egress is signaled as a Target in the P-DAO, and as the last
> entry is
>   1148	   an SF-VIO of a last segment towards that Egress.
>   1149
>   1150	   A Non-Storing-Mode P-DAO signals a strict or loose sequence
> of nodes
>                                    ^ contains an SR-VIO to...
>   1151	   between the Track Ingress (excluded) and a Track Egress
> (included).
>   1152	   It installs a source-routed path of a higher precedence
> within the
>                                                         ^^^^^^^^^^ same
> consideration as above

I agree that precedence was used with 2 meanings. 

The other text is changed to 

    Upon receiving a propagated DAO, all except the Egress router MUST install a
    route towards the DAO Target(s) via their successor in the SM-VIO. A router
    that cannot store the routes to all the Targets in a P-DAO MUST reject the
    P-DAO by sending a DAO-ACK to the Root with a Rejection Status of "Out of
    Resources" as opposed to forwarding the DAO to its predecessor in the list.
    The router MAY install additional routes towards the VIA Addresses that are
    the SM-VIO after self, if any, but in case of a conflict or a lack of
    resource, the route(s) to the Target(s) are the ones that must be installed
    in priority.


>   1153	   Track indicated by the P-DAO Base Object, towards the
> Targets
>   1154	   indicated in the Target Options.  The source-routed path
> requires a
>   1155	   Source-Routing header which implies an encapsulation to add
> the SRH
> 
> ^^^^^^^^^^^
> 
> Same consideration as much earlier, e.g.: encapsulate into new IPv4 with
> SRH ?!


IPv6 but yes. Added Ip in IP. 


>   1156	   to an existing packet.
>   1157
>   1158	   The next entry in the sequence must be either a neighbor of
> the
>   1159	   previous entry, or reachable as a Target via another
> Projected Route,
>   1160	   either Storing or Non-Storing.  If it is reachable over a
> Storing
>   1161	   Mode Projected Route, the next entry in the loose sequence
> is the
>   1162	   Target of a previous segment and the ingress of a next
> segment; the
>   1163	   segments are associated with the same Track, which avoids
> the need of
>   1164	   an encapsulation.  Conversely, if it is reachable over a
> Non-Storing
>   1165	   Mode Projected Route, the next loose source routed hop of
> the inner
>   1166	   Track is a Target of a previous Track and the ingress of a
> next
>   1167	   Track, which requires a de- and a re-encapsulation.
> 
> This is a game of blindfold chess. Pictured examples of the two cases
> described
> would help a lot to understand and verify the points made/claimed here.

Pointed on the examples, e.g.,


   If the next entry in the loose sequence is reachable over a Storing
   Mode P-Route, it MUST be the Target of a Segment and the Ingress of a
   next segment, both already setup; the segments are associated with
   the same Track, which avoids the need of an additional encapsulation.
   For instance, in Section 3.4.1.3, Segments A==>B-to-C and
   C==>D==>E-to-F must be installed with Storing-Mode P-DAO messages 1
   and 2 before the Track A-->C-->E-to-F that joins them can be
   installed with Non-Storing-Mode P-DAO 3.

   Conversely, if it is reachable over a Non-Storing Mode P-Route, the
   next loose source-routed hop of the inner Track is a Target of a
   previously installed Track and the Ingress of a next Track, which
   requires a de- and a re-encapsulation when switching the outer Tracks
   that join the loose hops.  This is examplified in Section 3.4.2.3
   where Non-Storing Mode P-DAO 1 and 2 install strict Tracks that Non-
   Storing Mode P-DAO 3 joins as a super Track.  In such a case, packets
   are subject to double IP-in-IP encapsulation.


> 
>   1181	   A Serial Track is installed by a single Projected Routes
> that signals
>                                                                   ^
>   1182	   the sequence of consecutive nodes, either in Storing or Non-
> Storing
>   1183	   Mode.  If can be a loose Non-Storing Mode Projected Route,
> in which
>                    ^
>   1184	   case the next loose entry must recursively be reached over a
> Serial
>   1185	   Track.
> 
> Please check whats standard *-Storing-Mode or *-Storing Mode - and make
> text
> consistent.
> 

Removed the - 

>   1186
>   1187	   A Complex Track can be installed as a collection of
> Projected Routes
>   1188	   with the same DODAGID and Track ID.  The Ingress of a Non-
> Storing
> 
> If this is turned around, does it become the still missing definition of a
> complex track ?

Now there's a whole section 

3.5.  Complex Tracks

Before that we have

   A Serial Track comprises provides only one path between Ingress and
   Egress.  It comprises at most one Leg. A Stand-Alone Segment defines
   implicitly a Serial Track from its Ingress to Egress.

   A complex Track forms a graph that provides a collection of potential
   paths to provide redundancy for the packets, either as a collection
   of Legs that may be parallel or cross at certain points, or as a more
   generic DODAG that is rooted at the Ingress Node.




> 
> Multiple projected routes can be installed into a single (DODAGID,TrackID).
> This leads to Sequential or Complex Tracks ??

Sequential if sequential else complex... I hope the definition above helps.

> 
>   1188	   with the same DODAGID and Track ID.  The Ingress of a Non-
> Storing
>   1189	   Mode Projected Route must be the owner of the DODAGID.  The
> Ingress
>   1190	   of a Storing Mode Projected Route must be either the owner
> of the
>   1191	   DODAGID, or the egress of a preceding Storing Mode Projected
> Route in
>   1192	   the same Track.  In the latter case, the Targets of the
> Projected
>   1193	   Route must be Targets of the preceding Projected Route to
> ensure that
>   1194	   they are visible from the track Ingress.
>   1195
>   1196	7.3.1.  Storing-Mode P-Route
> 
> Please go back through the text and replace all instances of Projected
> Route
> after the first occurance with P-Route. You use both terms randomnly
> interchanged,
> which is confusing. Better to stick to the abbreviation only.

OK; I used what seemed to ring in my ear.


> 
> I also observe, that i would be much less confused about the text if this
> high-level
> explanation was preceeding the detailed explanation earlier in the
> beginning of section 7.
> 
> Please think how to resort the text of section 7 so it starts with the
> pictures
> and high-level explanation and then goes into the more formal detail spec.

Done

> 
>   1198	   Profile 1 extends RPL operation in a Non-Storing Mode
> network with
> 
> Without further explanation, the first unexplained use of Profile is
> derailing and
> confusing. Suggest to just delete the mentioning of profiles and just
> define them
> in Section 8.

Done

>   1199	   Storing-Mode Projected Routes that install segments along
> the main
>   1200	   DODAG and enable to loose source routing between the Root
> and the
>   1201	   targets.
>   1202
>   1203	7.3.1.1.  Installing a Storing-Mode P-Route
>   1204
>   1205	   As illustrated in Figure 9, a P-DAO that carries a SF-VIO
> enables the
>   1206	   Root to install a stateful route towards a collection of
> Targets
>   1207	   along a Segment between a Track Ingress and a Track Egress
> using a
>   1208	   projected DAO Message.
>   1209
>   1210	           ------+---------
>   1211	                 |          Internet
>   1212	                 |
>   1213	              +-----+
>   1214	              |     | Border Router
>   1215	              |     |  (RPL Root)
>   1216	              +-----+                      |     ^
> |
>   1217	                 |                         | DAO | ACK
> |
>   1218	           o    o   o    o                 |     |
> |
>   1219	       o o   o  o   o  o  o o   o          |  ^       |
> Projected    .
>   1220	      o  o o  o o    o\  o   o  o  o       |  | DAO   | Route
> .
>   1221	      o   o    o  o    \o--o    o  o  o    | ^        |
> .
>   1222	     o  o   o  o   o        \o   o o       v | DAO    v
> .
>   1223	     o          o   LLN   T1   T2     o
> |
>   1224	         o o   o        o     o              Loose Source Route
> Path |
>   1225	      o       o      o    o                 From Root To
> Destination v
>   1226
>   1227	                        Figure 9: Projecting a route
> 
> 
> Do you think you can visualise an example P-route into the picture ?
> I started a bit of bit of it above..
> 
>   1228
>   1237	   In order to install the relevant routing state along the
> Segment ,
>   1238	   the Root sends a unicast P-DAO message to the Track Egress
> router of
>   1239	   the routing Segment that is being installed.  The P-DAO
> message
>   1240	   contains a SF-VIO with the direct sequence of Via Addresses.
> The SF-
>                                       ^^^^^^ strict

Done

> 
>   1241	   VIO follows one or more RTOs indicating the Targets to which
> the
>   1242	   Track leads.  The SF-VIO contains a Segment Lifetime for
> which the
> 
> In this explanation it would be helpful to explain conditions for targets,
> e.g.:
> Target must either be direc neighbor of the P-Route egres, or be Target of
> some
> P-Route2 with the P-Route egres as its ingres. Right ?

This is true and discussed prior 

> 
>   1243	   state is to be maintained.
>   1244
>   1245	   The Root sends the P-DAO directly to the egress node of the
> Segment.
>   1246	   In that P-DAO, the destination IP address matches the last
> Via
>   1247	   Address in the SF-VIO.  This is how the egress recognizes
> its role.
>   1248	   In a similar fashion, the ingress node recognizes its role
> as it
>   1249	   matches first Via Address in the SF-VIO.
>   1250
>   1251	   The Egress node of the Segment is the only node in the path
> that does
>   1252	   not install a route in response to the P-DAO; it is expected
> to be
>   1253	   already able to route to the Target(s) on its own.  If one
> of the
>   1254	   Targets is not known, the node MUST answer to the Root with
> a
>   1255	   negative DAO-ACK listing the Target(s) that could not be
> located
>   1256	   (suggested status 10 to be confirmed by IANA).
> 
> So complex/sequential Tracks need to be established from destination to
> source.
> What happens with a track when th egress looses routing information for a
> destination ?
> (if thats answere elsehwhere, a pointer to that section would be helpfull
> here).

If that's the forwarding error there's effectively text in the forwarding section.
Should not duplicate should we?

> 
>   1258	   If the egress node can reach all the Targets, then it
> forwards the
>   1259	   P-DAO with unchanged content to its loose predecessor in the
> Segment
> 
> Shouldn't that be strict instead of loose ? If it was loose, how would that
> loose
> predecessor know how to send the data packets to the successor without
> adding
> another encap/RH if the underlying main RPL instance was NSM ?

Correct, removed loose

> 
>   1260	   as indicated in the list of Via Information options, and
> recursively
>   1261	   the message is propagated unchanged along the sequence of
> routers
>   1262	   indicated in the P-DAO, but in the reverse order, from
> egress to
>   1263	   ingress.
>   1264
>   1265	   The address of the predecessor to be used as destination of
> the
>   1266	   propagated DAO message is found in the Via Address the
> precedes the
>   1267	   one that contain the address of the propagating node, which
> is used
>   1268	   as source of the message.
>   1269
>   1270	   Upon receiving a propagated DAO, all except the Egress
> Router MUST
>   1271	   install a route towards the DAO Target(s) via their
> successor in the
>   1272	   SF-VIO.  The router MAY install additional routes towards
> the VIA
>   1273	   Addresses that are the SF-VIO after the next one, if any,
> but in case
>   1274	   of a conflict or a lack of resource, the route(s) to the
> Target(s)
>   1275	   have precedence.
>   1276
>   1277	   If a router cannot reach its predecessor in the SF-VIO, the
> router
>   1278	   MUST answer to the Root with a negative DAO-ACK indicating
> the
>   1279	   successor that is unreachable (suggested status 11 to be
> confirmed by
>   1280	   IANA).
>   1281
>   1282	   The process continues till the P-DAO is propagated to
> ingress router
>                                  ^until
>   1283	   of the Segment, which answers with a DAO-ACK to the Root.
> The Root
>   1284	   always expects a DAO-ACK, either from the Track Ingress with
> a
>   1293	   positive status or from any node along the segment with a
> negative
>   1294	   status.  If the DAO-ACK is not received, the Root may retry
> the DAO
>   1295	   with the same TID, or tear down the route.
>   1296
>   1297	7.3.1.2.  Maintaining and Tearing Down a Storing-Mode P-Route
>   1298
>   1299	   A Segment Lifetime of 0 in a VIO is used to clean up the
> state.  The
>                                                                     ^
>                                                                     P-route
> state
>   1300	   P-DAO is forwarded as described above, but the DAO is
> interpreted as
>   1301	   a No-Path DAO and results in cleaning up existing state as
> opposed to
>   1302	   refreshing an existing one or installing a new one.
>   1303
>   1304	   Note that the continuity of the segment may be broken; this
> happens
>   1305	   if the bidirectional connectivity between contiguous hops of
> the
>   1306	   segment is lost.  In that case the Root needs to send the
> projected
>   1307	   DAO to one or more intermediate node(s) as opposed to the
> egress
>   1308	   node, indicating a portion of segment that is being replaced
> or
>   1309	   cleaned up.  At the extreme, the P-DAO updates only one
> node, in
>   1310	   which case it contains only one VIO.

> 
> Hmm... how does this work given how the egres is meant to not install route
> state,
> so if you want to fix up some intermediate segments, then the egres of that
> fixup VIO, which is not the final egres of the full VIO  would from the
> prior
> explanation have to invoke its function of checking the targets
> reachability,
> but not to establish storing mode state. Some more explanation here might
> be
> helpful. I think his can work, but i think the egres of a fixup VIO needs
> to
> be careful and behave differently to a non-fixup actual track egres.

That's the case where the P-DAO removes a state (as opposed to install) so yes that can be only one node, say being replaced by another.


>   1311
>   1312	   In case of a forwarding error along an Storing-Mode P-Route,
> the node
>   1313	   that fails to forward SHOULD send an ICMP error with a code
> "Error in
>   1314	   Projected Route" to the Root.  Failure to do so may result
> in packet
>   1315	   loss and wasted resources along the Projected Route that is
> broken.
>   1316
>   1317	7.3.2.  Non-Storing-Mode P-Route
>   1318
>   1319	   As illustrated in Figure 10, a P-DAO that carries an SR-VIO
> enables
>   1320	   the Root to install a source-routed path from a Track
> Ingress towards
>   1321	   a Target along the main DODAG.
>   1349
>   1349	              ------+---------
>   1350	                    |          Internet
>   1351	                    |
>   1352	                 +-----+
>   1353	                 |     | Border Router
>   1354	                 |     |  (RPL Root)
>   1355	                 +-----+                    |  P  ^ ACK
>   1356	                    |        Track          | DAO |
>   1357	              o    o   o  o  Ingress X      V     |   X
>   1358	          o o   o  o   o  o     o   X   o             X Source
>   1359	         o  o o  o o    o   o  o    X  o  o           X Routed
>   1360	         o   o    °  o     o   o   o X     o          X Segment
>   1361	        o  o   o  o   o  o    o  o     X Track        X
>   1362	           o  o  o  o             o     Egress
>   1363
>   1364	          o       o               o    o
>   1365	        o          o             o     o
>   1366	                                      destination
>   1367
>   1368	                          LLN
>   1369
>   1370	                 Figure 10: Projecting a Non-Storing Route
> 
> So there is some attempt here to visualize the P-Route, but it mentions
> destination
> and not target, so i am again confused about the difference of those
> terms...
> Would be good to show Target(s) in the picture.

Done

> 
>   1372	   When forwarding a packet to a destination for which the
> router
> 
> which router ? The Track ingres ?


> 

Changed to


   When forwarding a packet to a destination for which a router determines
   that routing happens via a Track for which it is Ingress, the router must
   encapsulated the packet using IP-in-IP to add the Source Routing Header 
   with the final destination set to the Track Egress.


>   1373	   determines that routing happens via a Track Target, the
> router
> 
> "via a track target" sounds as if destination is different from trac
> target.
> What is the routing information from which that router can determine that
> a destination is reachable via a target ? The P-Route only signals targets,
> right ?

Right: "target" was extraneous

> 
>   1374	   inserts the Source Routing Header in the packet with the
> final
>   1375	   destination at the Track Egress.
> 
> encapsulates the packet with a new IPv6/SRH header ?
> 
> Now it sounds as if final destination is the same as track egres, so not
> even target...
> even more confused now.

The outer destination (of IP in IP) is the Track egress. Which decapsulates and forwards to the Target.

>   1376
>   1377	   In order to signal the Segment, the router encapsulates the
> packet
>   1378	   with an IP-in-IP header and a Routing Header as follows:
> 
> It seems as if we're not discussing installation of non-storing P-Routes at
> all,
> but immediately skip to data-plane for non-storing P-Routes ?
> 

You right, I move all that text to a section " Encapsulating and Forwarding Along a Track "
After installing a Track.

> I would suggest to have even if a very short section for installation of
> the non-storing
> P-Route in one subsection and then aother subsection for the data-plane.
> 
> The storing mode text above does not seem to have a dedicated data-plane
> subsecton too,
> so quite inconsistent...

Done



> 
>   1379
>   1380	   *  In the uncompressed form the source of the packet is this
> router,
> 
> this router...
> 

Aka self, doesn't it read ok?


*     In the uncompressed form the source of the packet is the address
      that this router uses as DODAGID for the Track, the destination is
      the first Via Address in the NSM-VIO, and the RH is a Source
      Routing Header (SRH) [RFC6554] that contains the list of the
      remaining Via Addresses terminating by the Track Egress.



>   1381	      the destination is the first Via Address in the SR-VIO,
> and the RH
>   1382	      is a Source Routing Header (SRH) [RFC6554] that contains
> the list
>   1383	      of the remaining Via Addresses terminating by the Track
> Egress.
>   1384
>   1385	   *  The preferred alternate in a network where 6LoWPAN Header
>   1386	      Compression [RFC6282] is used is to leverage "IPv6 over
> Low-Power
>   1387	      Wireless Personal Area Network (6LoWPAN) Paging Dispatch"
>   1388	      [RFC8025] to compress the RPL artifacts as indicated in
> [RFC8138].
>   1389
>   1390	      In that case, the source routed header is the exact copy
> of the
>   1391	      (chain of) SRH-6LoRH found in the SR-VIO, also
> terminating by the
>   1392	      Track Egress.  The RPI-6LoRH is appended next, followed
> by an IP-
>   1393	      in-IP 6LoRH Header that indicates the Ingress Router in
> the
>   1394	      Encapsulator Address field, see as a similar case Figure
> 20 of
>   1395	      [TURN-ON_RFC8138].
> 
> I will trust this is correct, as i can not verify this quickly due to my
> lack of
> this encap details.

Now we have 

  6.7.  Compression of the RPL Artifacts

> 
>   1404
>   1405	   In the case of a loose source-routed path, there MUST be
> either a
>   1406	   segment for the same Track to the loose next hop, on which
> case the
>   1407	   packet is forwarded to the next hop along that segment, a
> common
>   1408	   neighbor with the loose next hop, on which case the packet
> is
>   1409	   forwarded to that neighbor, or another Track to the loose
> next hop
>   1410	   for which this node or a neighbor is Ingress.  In the latter
> case,
>   1411	   another encapsulation takes place and the process possibly
> recurses;
>   1412	   otherwise the packet is dropped.
> 
> I can not comment on this because i am still lost as to the semantic of
> segment given
> he absence of any illustrative pictures for multi-segment Tracks so far in
> the document.

Done

> 
>   1414	   In case of a forwarding error along a Source Route path, the
> node
>   1415	   that fails to forward SHOULD send an ICMP error with a code
> "Error in
>   1416	   Source Routing Header" back to the source of the packet, as
> described
>   1417	   in section 11.2.2.3. of [RPL].  Upon this message, the
> encapsulating
>   1418	   node SHOULD stop using the source route path for a period of
> time and
> 
> How long should that period be. Elaborate for how the encapsulating node
> should determine the period.

Depends heavily on deployment. RPL does not specify thos  for that reason.

Updated

"                          SHOULD stop using the source route path for a
      reasonable period of time which duration depends on the deployment "

> 
>   1419	   it SHOULD send an ICMP message with a Code "Error in
> Projected Route"
>   1420	   to the Root.  Failure to follow these steps may result in
> packet loss
>   1421	   and wasted resources along the source route path that is
> broken.
>   1422
>   1423	7.4.  Forwarding Along a Track
> 
> In 7.3.2 you already detail parts of this. Again, i would strongly suggest
> to revert order, with this section first, before getting into the currently
> earlier stated details.


Done

> 
>   1425	   This draft leverages the RPL Forwarding model follows:
>   1426
>   1427	   *  In the data packets, the Track DODAGID and the TrackID
> MUST be
>   1428	      respectively signaled as the IPv6 Source Address and the
>   1429	      RPLInstanceID field of the RPI that MUST be placed in the
> outer
>   1430	      chain of IPv6 Headers.
> 
> I think an outer chain(mail) can be found as an armor on medieval knights,
> but in our
> case i think it is the outer header of the packets chain of IPv6 headers.

😊  😊  😊 
> 
>   1432	      The RPI carries a local RPLInstanceID called the TrackID,
> which,
>   1433	      in association with the DODAGID, indicates the Track
> along which
>   1434	      the packet is forwarded.
>   1435
>   1436	      The D flag in the RPLInstanceID MUST be set to 0 to
> indicate that
>   1437	      the source address in the IPv6 header is set ot the
> DODAGID, more
>                                                            ^^
>   1438	      in Section 7.4.
> 
> This is section 7.4.


Fixed


> 
>   1439
>   1440	   *  This draft conforms the principles of [USEofRPLinfo] with
> regards
>                                  ^ to
>   1441	      to packet forwarding and encapsulation along a Track.
>   1442
Ok

>   1443	      -  In that case, the Track is the DODAG, the Track
> Ingress is the
> 
> In the case where the Track is the main DODAG ... ?

Rather "With this draft,"

> 
>   1444	         Root, and the Track Egress is a RAL, and neighbors of
> the Track
> 
> Is there any reason why to say RAL instead of node ? First time you use
> RAL, so wondering.

Changed to

      the Track is the DODAG, the Track Ingress is the Root, and the Track Egress
      is a RPL-Aware Node (RAN), and neighbors of the Track Egress that can be 
      reached via the Track are RPL-Unaware Leaves (RULs)


> 
>   1445	         Egress that can be reached via the Track are RULs.
> The
>   1446	         encapsulation rules in [USEofRPLinfo] apply.
> 
> Not sure what RFC editor says, but given how RAL/RUL are first mentioned
> here,
> even though they are listed in glossary, expanding them here can not hurt.
> 
> I think you may want to find a place early in he document where you want to
> say
> that "node" without qualification in this doc mean RAN, maybe also add to
> glossary.
> 
> It would be helpfull to clariy what i think this section wants to say
> without hoping that
> readers can / want to deduce it from [UseofRPLinfo]. How about the
> following table:
> 
>         Node role                   Supported node type
>         Track ingres                RAN
>         Track midpoint              RAN
>         Track egres                 RAN, RAL
>         Neighbor of track egres     RAN, RAL, RUL
> 
> (hope this is roughly ok, else fix up).

It is OK, feels like overkill when the point is really that the Targets are RULs for the Track perspective (as a DODAG)	.

Slightly reworded

      With this draft, the Track is a RPL DODAG. From the perspective of that
      DODAG, the Track Ingress is the Root, the Track Egress is a RPL-Aware
      6LR, and neighbors of the Track Egress that can be reached via the Track,
      but are external to it, are external destinations and treated as
      RPL-Unaware Leaves (RULs).

> 
>   1448	      -  If the Track Ingress is the originator of the packet
> and the
>   1449	         Track Egress is the destination of the packet, there
> is no need
>   1450	         for an encapsulation.
>   1451
>   1459
>   1460
>   1461	      -  So the Track Ingress must encapsulate the traffic that
> it did
>   1462	         not originate, and add an RPI in any fashion.
>   1463
>   1464	      A packet that is being routed over the RPL Instance
> associated to
>   1465	      a first Non-Storing Mode Track MAY be placed
> (encapsulated) in a
>   1466	      second Track to cover one loose hop of the first Track.
> On the
> 
> I can not follow this explanation without an example picture. When reading
> it,
> the first thing that comes to mind is that it sounds as if i could have
> sequential
> second Tracks in different RPL Instances, but it is unclear whether this is
> is a predondition of this case of whether its irrelevant.

Added a reference to the non storing segment routing description 


> 
>   1466	      second Track to cover one loose hop of the first Track.
> On the
>   1467	      other hand, a Storing Mode Track must be strict and a
> packet that
>   1468	      it placed in a Storing Mode Track MUST follow that Track
> till the
> 
> :%s/\<till\>/until/g
> Don't speak emacs.
> 

https://www.grammarly.com/blog/until-till-til/


>   1469	      Track Egress.
>   1470
>   1471	      When a Track Egress extracts a packet from a Track
> (decapsulates
>   1472	      the packet), the Destination of the inner packet MUST be
> either
>   1473	      this node or a direct neighbor, or a Target of another
> Segment of
>   1474	      the same Track for which this node is ingress, otherwise
> the
>   1475	      packet MUST be dropped.
> 
> .... and (see reference) ICMP must be raied according to... etc. pp ?!

Added a next para on this right after

   In case of a failure forwarding a packet along a Segment, e.g., the
   next hop is unreachable, the node that discovers the fault MUST send
   an ICMPv6 Error message [RFC4443] to the Root, with a new Code "Error
   in P-Route" (See Section 10.13).  The Root can then repair by
   updating the broken Segment and/or Tracks, and in the case of a
   broken Segment, remove the leftover sections of the segment using SM-
   VIOs with a lifetime of 0 indicating the section ot one or more nodes
   being removed (See Section 6.5).


> 
> This last paragraph should really be as early in the document as possible.

I believe it's better now

> 
>   1476
>   1477	   All properties of a Track operations are inherited form the
> main RPL
>   1478	   Instance that is used to install the Track.  For instance,
> the use of
>   1479	   compression per [RFC8138] is determined by whether it is
> used in the
>   1480	   main instance, e.g., by setting the "T" flag [TURN-
> ON_RFC8138] in the
>   1481	   RPL configuration option.
> 
> Can there be multiple main instances ? If so, that should be mentioned and
> maybe an example given.

No, there's only one Main DODAG, changing all to that formulation.

> 
>   1482
>   1483	8.  Profiles
>   1484
>   1485	   This document provides a set of tools that may or may not be
> needed
>   1486	   by an implementation depending on the type of application it
> serves.
>   1487	   This sections described profiles that can be implemented
> separately
>   1488	   and can be used to discriminate what an implementation can
> and cannot
>   1489	   do.
>   1490
>   1491	   Profile 0  Profile 0 is the legacy support of [RPL] Non-
> Storing Mode.
>   1492	      It provides the minimal common functionality that must be
>   1493	      implemented as a prerequisite to all the Track-supporting
>   1494	      profiles.  The other Profiles extend Profile 0 with
> selected
>   1495	      capabilities that this specification introduces on top.
> 
> Is this profile a term estblished outside this document already, then
> please
> do provide reference. If it is only introuced in this document, then what
> exactly needs to be implemented for it ?
> 
> Let me guss: This is a new definition in this document, and it refers to
> nodes
> that do not implement anything from this draft, but only RPL NSM acccording
> to
> (fill in, best with section), and to deploy any of this documents
> technology,
> ALL RAL in the network MUST implement Profile 0 or better. Or maybe not...
> Maybe only RAL that are used to pass P-Route traffic MUST be Profile 0 or
> better ?!
> 
> 
> If i am guessing right than my prior paragraphs text might be a better
> starting
> point with less gussing.

Added 

    This document provides a set of tools that may or may not be needed by
    an implementation depending on the type of application it serves.
    This sections described profiles that can be implemented separately and
    can be used to discriminate what an implementation can and cannot do.
    This section describes profiles that enable to implement only a portion
    of this specification to meet a particular use case. Only Profiles 0 and 1
    are REQUIRED by all implementations; all the other profiles are OPTIONAL

Note that profile 0 is RPL NSM per RFC 6550, as is. That's why it's called zero.

> 
>   1496
>   1497	   Profile 1 (Storing-Mode P-Route Segments along the main
> DODAG)  Profi
>   1498	      le 1 does not create new paths; it combines Storing and
> Non-
>   1499	      Storing Modes to balance the size of the routing header
> in the
>   1500	      packet and the amount of state in the intermediate
> routers in a
>   1501	      Non-Storing Mode RPL DODAG.
> 
> You should be able to state exactly what part of this spec a node
> MUST/SHOULD
> implement to be in compliance with Profile 1. Same is true for the other
> profiles.
> Might be difficult to structure the documents into subsections such that
> you can list those subsections that are required (incrementally) for each
> Profile,
> but otherwise its really hard to figure out if or if not an implementation
> is
> compliant with a profile.
 

Effectively that would be quite impractical to do. 


>   1503	   Profile 2 (Non-Storing-Mode P-Route Segments along the main
> DODAG)  P
>   1504	      rofile 2 extends Profile 0 with Strict Source-Routing
> Non-Storing-
> 
> Can you try to persuade tools to not split words (P rofile) ?

Sorry for that. Only the text version I dare hope

> 
>   1505	      Mode Projected Routes along the main DODAG.  Profile 2
> provides
>   1506	      the same capability to compress the SRH in packets down
> the main
>   1507	      DODAG as Profile 1, but it require an encapsulation, in
> order to
>   1508	      insert an additional SRH between the loose source routing
> hops.
>   1509
>   1517	   Profile 3  Profile 3 and above build Tracks that do not
> necessarily
>   1518	      follow the main DODAG.  In order to form the best path
> possible,
> 
> ^ ?
> DODAG ? "best possible" is unexplained. Might be good to give (maybe
> textual is
> sufficient) the most simple example of how an additional DODAG/TrackID
> would
> be limited if it can not support Sibling Information Option. This might be
> obvious to RPL experts of course..

Added early in the doc

    This specification expects that the RPL Main DODAG is operated in RPL
    Non-Storing Mode to sustain the exchanges with the Root. Based on its
    comprehensive knowledge of the parent-child relationship, the Root can form
    an abstracted view of the whole DODAG topology. This document adds the
    capability for nodes to advertise additional sibling information to
    complement the topological awareness of the Root to be passed on to the PCE,
    and enable the PCE to build more / better paths that traverse those siblings.
> 
>   1519	      those Profiles require the support of Sibling Information
> Option
>   1520	      to inform the Root of additional possible hops.  Profile
> 3 extends
>   1521	      Profile 1 with additional Storing-Mode Projected Routes
> that
>   1522	      install segments that do not follow the main DODAG.  If
> the
>   1523	      Segment Ingress (in the SF-VIO) is the same as the IPv6
> Address of
>   1524	      the Track Ingress (in the projected DAO base Object), the
> P-DAO
>   1525	      creates an implicit Track between the Segment Ingress and
> the
>   1526	      Segment Egress.
>   1527
>   1528	   Profile 4  Profile 4 extends Profile 2 with Strict Source-
> Routing
>   1529	      Non-Storing-Mode Projected Routes to form Tracks inside
> the main
>   1530	      DODAG.  A Track is formed as one or more strict source
> source
>   1531	      routed paths between the Root that is the Track Ingress,
> and the
>   1532	      Track Egress that is the last node
> 
> Why is this Profile 4 when Profile 3 says that all further profiles
> do not necessarily follow the main DODAG ? Aka: What would not work if
> you where to make 4 become eg.: 2.5 ?

The Track is inside the DODAG but does not follow it. like east west vs north south.

> 
>   1534	   Profile 5  Profile 5 Combines Profile 4 with Profile 1 and
> enables to
>   1535	      loose source routing between the Ingress and the Egress
> of the
>   1536	      Track.  As in Profile 1, Storing-Mode Projected Routes
> connect the
>   1537	      dots in the loose source route.
>   1538
>   1539	   Profile 6  Profile 6 Combines Profile 4 with Profile 2 and
> also
>   1540	      enables to loose source routing between the Ingress and
> the Egress
>   1541	      of the Track.
> 
> How about a table where rows are features, columns profiles and
> intersections
> checkmarks ? Right now i am lost in the details.

Hard to summarize in even less, I added text to clarify the NSM operation in the main DODAG that serves for profile 2.

   Profiles 0 to 2 operate in the Main RPL Instance and do not require
   the support of local RPL Instances or the indication of the RPL
   Instance in the data plane.  Profile 3 and above leverage Local RPL
   Instances to build arbitrary Tracks rooted at the Track Ingress and
   using its namespace for TrackID.

   Profiles 0 and 1 are REQUIRED by all implementations that may be used
   in LLNs; this enables to use Storing Mode to reduce the size of the
   Source Route Header in the most common LLN deployments.  Profile 2 is
   RECOMMENDED in high speed / wired environment to enable traffic
   Engineering and network automation.  All the other profile /
   environment combinations are OPTIONAL.
> 
>   1542
>   1543	9.  Example Track Signaling
>   1544
>   1545	   The remainder of the section provides an example of how a
> Track can
>   1546	   be signaled
>   1547
>   1548	                                                  ===> F
>   1549	                   A ===> B ===> C ===> D===> E <
>   1550	                                                  ===> G
>   1551
>   1552
>   1553	                         Figure 11: Reference Track
>   1554
>   1555	   A is Track ingress, E is track Egress.  C is stitching
> point.  F and
> 
> First time "stitching point" is used. Can you avoid a new term here ? Else
> explain.
> 
>   1556	   G are E's neighbors, "external" to the Track, and reachable
> from A
>   1557	   over the Track A->E.
> 
> I guess this is a two-segment Track with one segment A-...>C and one
> segment C-...>E.
> 
> What is the "<" after E good for ?


Ascii art for a fork. Does this look better?

                                                 /===> F
                   A ===> B ===> C ===> D===> E <
                                                 \===> G


> 
> Why use the same symbol ===> for track and for getting from E to F, G ?
> Are F and G  Targets and/or destinations ? Please introduce those terms
> into the example.
> 
> How about indicating both the physical connectivity with "---" and the
> track with "===>"

I'd rather signal strict vs. loose with this
Added

   Conventionally we use ==> to represent a strict hop and --> for a
   loose hop.  We use -to- like in C==>D==>E-to-F to represent coma-
   separated Targets, e.g., F is a Target for Segment C==>D==>E.  In
   this example, A is Track Ingress, E is Track Egress.  C is a
   stitching point.  F and G are "external" Targets for the Track, and
   become reachable from A via the Track A-->E-to-F,G.




> 
> Also show / disinguish the two segments accordingly through better ASCII
> graphics.
> 
>   1558
>   1559	   In a general manner we want:
>   1560
>   1561	   *  P-DAO 1 signals C==>B==>E
> 
> I hope B should be D, else i am heck more confused.


Typo, I had already debunked that one but good catch


> 
> Please say what this signaling establishes. We learned in before that a P-
> DAO
> has not only a VIO, but also some targets. So what are the targets of this
> P-DAO ?
> 
>   1562
>   1563	   *  P-DAO 2 signals A==>B==>C
> 
> Likewise

For each of the 6 cases I added something like


   In this formulation:

   *  P-DAO 1 signals C==>D==>E-to-E,F,G

   *  P-DAO 2 signals A==>B==>C-to-E,F,G

> 
>   1573	   *  P-DAO 3 signals F and G via the A==>E Track
> 
> What does hthis mean ? Whats the entry, whats the exit ? whats the VIO,
> whats the
> target of P-DAO 3 ?

Which the new formulation just above hopefully explains better; I reformulate again the text as

Conventionally we use ==> to represent a strict hop and --> for a
   loose hop.  We use -to- like in C==>D==>E-to-F to represent coma-
   separated Targets, e.g., F is a Target for Segment C==>D==>E.  In
   this example, A is Track Ingress, E is Track Egress.  C is a
   stitching point.  F and G are "external" Targets for the Track, and
   become reachable from A via the Track A(ingress) to E (Egress and
   implicit Target) leading to F and G (explicit Targets).

   In a general manner the desired outcome is as follows:

   *  Targets are E, F, and G

   *  P-DAO 1 signals C==>D==>E

   *  P-DAO 2 signals A==>B==>C

   *  P-DAO 3 signals F and G via the A-->E Track

> 
>   1575	   P-DAO 3 being loose, it can only be non-storing.  Note that
> since the
>   1576	   Root is always the ingress of a Track, and all SR-VIOs are
> now Track,
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^
> ^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Which root ? the root of the main DODA ?
> I don't parse the second ^^^^ either.


Reworded as

   Loose sequences of hops must be expressed in Non-Storing Mode, so
   P-DAO 3 contains a NSM-VIO.  With this specification, the DODAGID to
   be used by the Ingress as source address is signaled if needed in the
   DAO base object, the via list starts at the first loose hop and
   matches the source route header, and the Egress of a Non-Storing Mode
   P-DAO is an implicit Target that is not listed in the RTO.
> 
>   1577	   the Root being signaled in the DAO base object can now be
> elided in
>   1578	   the VIA list in SR-VIO.  This enables the construction by
> the main
>   1579	   root of the RFC 8138 optimized SRH-6LoRH in the SR-VIO, to
> be placed
>   1580	   as is in the packet by the Root.
> 
> I can not follow that paragraph. Would it be posible to constrain the
> example
> to not also include the SRH-6LoRH complexity, given how its an option ? One
> step at a time ?
> 
Removed from there. 

I looked again at the text based on this comment and ended as

   When the [RFC8138] compression is used, the Root of the Main DODAG
   that sets up the Track also constructs the compressed routing header
   (SRH-6LoRH) on behalf of the Track Ingress, which saves the
   complexities of optimizing the SRH-6LoRH encoding in constrained
   code.  The SRH-6LoRH is signaled in the NSM-VIO, in a fashion that it
   is ready to be placed as is in the packet encapsulation by the Track
   Ingress.


>   1581
>   1582	9.1.  Using Storing-Mode Segments
>   1583
>   1584	   A==>B==>C and C==>D==>E are segments of a same Track.  Note
> that the
> 
> See above. his explanation comes too late.
> 
>   1585	   storing mode signaling imposes strict continuity in a
> segment.  One
>            ^^^^^^^^^^^^^^^^^^^^^^                ^^^^^^^^^^ steering
> 
> So i am confused if actually the storing mode signaling imposes that strict
> steering. I guess if i start from the egres, and it wants to forward the P-
> DAO
> to the prior hop node in the VIO, if that prior hop is a neighbor, then it
> would send the P-DAO directly to that neighbor, but if it is not a direct
> neighbor, it would need to go through the root. I guess this is 101
> knowledge
> from RFC6550, so if i had the time to read anothe 150 pages, i might be
> able
> to deduce that, but i guess it wouldn't hurt to put the same explanation
> into
> th text, e.g.: P-DAO must be sent directly from each segment node to its
> prior segment node, and in a non-storing DODAG, this is only possible if
> they are neighbors.

All correct

Recorded to add that as


    Note that the Storing Mode signaling imposes strict continuity in a 
    segment, since the P-DAO is passed hop-by-hop, as a classical DAO is,
    along the reverse datapath that it signals. One benefit of strict
    routing is that loops are avoided along the Track.

> 
> Also:
> One of the fundamental explanations missing is the definition of the
> relationship
> between a segment and a P-Route. The way i figure, a P-DAO is (segment,
> {targets})

Yes

> and represents a set of P-routes { targetI -> segment } targetI in
> {targets}).
> 
Yes 



> And then definition of tracks as composed of sequences of P-routes
> constructed
> from likely trees of P-DAO
> 
> Or something like that...

Not really trees, more likely interlaced or parallel lines from one ingress to usually one egress.
We're missing the capability to signal north south segments that join lines.
I'm adding text to enable the signaling of a real DODAG when the nodes are not constrained and can implement more Root functionality to build their own DODAG and select  paths

7.2.  A Track as a Full DODAG

   This specification builds parallel or crossing Track Legs as opposed
   to a more complex DODAG with interconnections at any place desirable.
   The reason for that limitation is related to constrained node
   operations, and capability to store large amount of topological
   information and compute complex paths:

   *  With this specification, the node in the LLN has no topological
      awareness, and does not need to maintain dynamic information about
      the link quality and availability.

   *  The Root has a complete topological information and statistical
      metrics that allow it or its PCE to perform a global optimization
      of all Tracks in its DODAG.  Based on that information, the Root
      computes the Track Leg and predigest the source route paths.

   *  The node merely selects one of the proposed paths and applies the
      associated pre-computed routing header in the encapsulation.  This
      alleviates both the complexity of computing a path and the
      compressed form of the routing header.

   The "Reliable and Available Wireless (RAW) Architecture/Framework"
   [RAW-ARCHI] actually expects the PSE at the Track Ingress to react to
   changes in the forwarding conditions along the Track, and reroute
   packets to maintain the required degree of reliability.  To achieve
   this, the PSE need the full richness of a DODAG to form any path that
   could make meet the SLAs.

   This section specifies the additions that are needed to turn the
   Track into a full DODAG and enable the main Root to provide the
   necessary topological information to the Track Ingress.  The
   expectation is that the metrics that the PSE uses are of an order
   other than that of the PCE, because of the difference of time scale
   between routing and forwarding, mor e in [RAW-ARCHI].  It follows
   that the PSE will learn the metrics it needs from an alternate
   source, e.g., OAM frames.

   To pass the topological information to the Ingress, the Root uses a
   P-DAO messages that contains sequences of Target and Transit
   Information options that collectively represent the Track, expressed
   in the same fashion as in classical Non-Storing Mode.  The difference
   as that the Root is the source as opposed to the destination, and can
   report information on many Targets, possibly the full Track, with one
   P-DAO.

   Note that the Path Sequence and Lifetime in the TIO are selected by
   the Root, and that the Target/Transit information tupples in the
   P-DAO are not those received by the Root in the DAO messages about
   the said Targets.  The Track may follow sibling routes and does not
   need to be congruent with the Main DODAG.

> 
> 
>   1586	   benefit of strict routing is that loops are avoided along
> the Track.
> 
> As long as the underlying topology does not change and direct neigbors
> start
> to becom non-diret neighbors. In which case i think the segment wold need
> to be immediately invalidated. But the question raised in before about
> ICMP errors is valid, especially when i think about path divesity
> forwarding.
> In path diversity you often do NOT want to have traffic triggered errors,
> but
> jus sit out the error, even if that trafic gets dropped somehwere along the
> path.

True

Not every time but at some point you do (when it gets permanent). 
Note that wireless links vary a lot and at RAW we use OAM to collect the statistics and report to the main Root.



> Any desirable repair to do on the topolocy could/should come ffom non-
> traffic
> trigers, such as neighbor tracking to the management plane.
> 
> Is there any way for traffic itself in the Routing information header to
> indicate
> if it does or does not want to get ICMP indications when it fails to get
> forwarded ?
> That would be ideal to distingish between non-redundant and redundant
> traffic
> ICMP reaction...

I clarify in the ICMP text that it is only upon permanent errors, but the 
Concept of permanent must remain abstract.

> 
>   1587
>   1588	9.1.1.  Stitched Segments
> 
> Sequential ?
> 
>   1589
>   1590	   Storing-Mode P-DAO 1 and 2 are sent to E and C, as follows:
>   1591
>   1592	              +===============+==============+==============+
>   1593	              |     Field     | P-DAO 1 to E | P-DAO 2 to C |
>   1594	              +===============+==============+==============+
>   1595	              |      Mode     | Storing      | Storing      |
>   1596	              +---------------+--------------+--------------+
>   1597	              | Track Ingress | A            | A            |
>   1598	              +---------------+--------------+--------------+
>   1599	              |    TrackID    | (A, 129)     | (A, 129)     |
>   1600	              +---------------+--------------+--------------+
>   1601	              |      VIO      | C, D, E      | A, B, C      |
>   1602	              +---------------+--------------+--------------+
>   1603	              |    Targets    | E, F, G      | E, F, G      |
>   1604	              +---------------+--------------+--------------+
>   1605
>   1606	                          Table 1: P-DAO Messages
> 
> Please add a line showing the SegmentID of the VIO so its clear how

If that helps..



              +===============+==============+==============+
              |     Field     | P-DAO 1 to E | P-DAO 2 to C |
              +===============+==============+==============+
              |      Mode     | Storing      | Storing      |
              +---------------+--------------+--------------+
              | Track Ingress | A            | A            |
              +---------------+--------------+--------------+
              |    TrackID    | (A, 129)     | (A, 129)     |
              +---------------+--------------+--------------+
              |   SegmentID   | 1            | 2            |
              +---------------+--------------+--------------+
              |      VIO      | C, D, E      | A, B, C      |
              +---------------+--------------+--------------+
              |    Targets    | E, F, G      | E, F, G      |
              +---------------+--------------+--------------+

> the P-DAO are distinguished. I guess that is the identifier used, right ?



Right, and it is significant for a Track ID so in NSM it can be reused as follows:


      +===============+==============+==============+==============+
      |               | P-DAO 1 to C | P-DAO 2 to A | P-DAO 3 to A |
      +===============+==============+==============+==============+
      |      Mode     | Non-Storing  | Non-Storing  | Non-Storing  |
      +---------------+--------------+--------------+--------------+
      | Track Ingress | C            | A            | A            |
      +---------------+--------------+--------------+--------------+
      |    TrackID    | (C, 131)     | (A, 129)     | (A, 141)     |
      +---------------+--------------+--------------+--------------+
      |   SegmentID   | 1            | 1            | 1            |
      +---------------+--------------+--------------+--------------+
      |      VIO      | D, E         | B, C         | E            |
      +---------------+--------------+--------------+--------------+
      |    Targets    | E            | E            | F, G         |
      +---------------+--------------+--------------+--------------+

> 
> If this example is pimped up as asked for above, it would make a good
> intro example, although it does not cover all options. But at last the
> fact that the track ingres for the secnd (C,D,E) segment is still A
> was not clear to me in before.
> 
> Also, i guess that if the track was more of a tree, then the targets
> of some earlier segments wold be a superset of the targets of later
> segments at a branching point.
> 
>   1608	   As a result the RIBs are set as follows:
>   1609
>   1629
> +======+=============+=========+=============+==========+
>   1630	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   1631
> +======+=============+=========+=============+==========+
>   1632	         |  E   | F, G        | P-DAO 1 | Neighbor    | (A,
> 129) |
> 
> But E would know about F and G alrady before P-DAO 1, so i wonder...
> 
> he example didn't call out which DODAG we're forwarding across. I guess
> this
> is a sequental Track, which the example also does not say.

It is illustrated at the beginning

Before diving deeper into Track Legs and Segments signaling and
   operation, this section provides examples of what how route
   projection works through variations of a simple example.  Say we want
   to build a Serial Track from node A to E in Figure 1, so A can route
   packets to either E or its neighbors F and G along A, B, C, D and E
   as opposed to via the Root:

                                                 /===> F
                   A ===> B ===> C ===> D===> E <
                                                 \===> G


                         Figure 1: Reference Track

would be good to
> say Destination in which context in the table. "Destination in RPLInstance
> TrackID1 ???"

That's a but heavy

> 
> Q: If this is not in the main DODAG, but in that TrackID1 DODAG,
> and we didn't send any P-DAO yet. Can we actually send/deliver packets ?

Without the Track the packets go along the main DODAG, IOW in NSM via the Root that adds an SRH

> I guess no, right ? So what seems to happen is that we first have some
> F, G neighbor information from the main DODAG, and because of P-DAO 1,
> this information is inherited into the forwarding for TrackID1 DODAG ?
> That would be good to show, or explain.

We could write a book, but in an RFC we end up scrubbing that sort of text.
A Segment installed from egress and connects to the target even as it is being built.
A Track Leg cannot be set up before all the segments it uses. So we're good.

> 
> Also, given how you previously whee pointing out that these routes
> are used over routes to the node because of prefix-length, should the
> entries in the destination not rather read F/128, G/128, likewise for all
> the
> other entries below ? Or does RPL imply fixed /128 unless its a /0 ?
> default
> route to the root ?

Agreed I only give /128 routes in the examples. RPL does not imply that though.


Adding 

   In this simple example we show host routes though RPL Targets can be prefixes.  

> 
> Also: TrackID is 129 i think. (A, 129) seems to be the DODAG (Identifier?).
> So maybe rename or rewrite accordingly.

OK


           +====================+==============+==============+
           |       Field        | P-DAO 1 to E | P-DAO 2 to C |
           +====================+==============+==============+
           |        Mode        | Storing      | Storing      |
           +--------------------+--------------+--------------+
           |   Track Ingress    | A            | A            |
           +--------------------+--------------+--------------+
           | (DODAGID, TrackID) | (A, 129)     | (A, 129)     |
           +--------------------+--------------+--------------+
           |     SegmentID      | 1            | 2            |
           +--------------------+--------------+--------------+
           |        VIO         | C, D, E      | A, B, C      |
           +--------------------+--------------+--------------+
           |      Targets       | E, F, G      | E, F, G      |
           +--------------------+--------------+--------------+


> 
>   1633	         +------+-------------+---------+-------------+--------
> --+
>   1634	         |  D   | E           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1635	         +------+-------------+---------+-------------+--------
> --+
>   1636	         |  "   | F, G        | P-DAO 1 | E           | (A,
> 129) |
>   1637	         +------+-------------+---------+-------------+--------
> --+
>   1638	         |  C   | D           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1639	         +------+-------------+---------+-------------+--------
> --+
>   1640	         |  "   | E, F, G     | P-DAO 1 | D           | (A,
> 129) |
>   1641	         +------+-------------+---------+-------------+--------
> --+
>   1642	         |  B   | C           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1643	         +------+-------------+---------+-------------+--------
> --+
>   1644	         |  "   | E, F, G     | P-DAO 2 | C           | (A,
> 129) |
>   1645	         +------+-------------+---------+-------------+--------
> --+
>   1646	         |  A   | B           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1647	         +------+-------------+---------+-------------+--------
> --+
>   1648	         |  A   | E, F, G     | P-DAO 2 | B           | (A,
> 129) |
>   1649	         +------+-------------+---------+-------------+--------
> --+
>   1650
>   1651	                            Table 2: RIB setting
>   1652
>   1653	   E recognizes that it is the Track Egress because it is both
> a Target
>   1654	   and a Segment Endpoint.
> 
> ... of P-DAO 1 ?

I removed that. No need.

> 
> What would happen if E was not listed as a Target in P-DAO 2 ?

OK that can be avoided if E is not an intended Target. I actually changed to represent that case.


           +====================+==============+==============+
           |       Field        | P-DAO 1 to E | P-DAO 2 to C |
           +====================+==============+==============+
           |        Mode        | Storing      | Storing      |
           +--------------------+--------------+--------------+
           |   Track Ingress    | A            | A            |
           +--------------------+--------------+--------------+
           | (DODAGID, TrackID) | (A, 129)     | (A, 129)     |
           +--------------------+--------------+--------------+
           |     SegmentID      | 1            | 2            |
           +--------------------+--------------+--------------+
           |        VIO         | C, D, E      | A, B, C      |
           +--------------------+--------------+--------------+
           |      Targets       | F, G         | F, G         |
           +--------------------+--------------+--------------+

In fact all the next hops in the VIO are implicit targets though it may cause the node to create too much state. This is why I had that "precedence" text.
I changed the term to priority to avoid the confusion you pointed out.

Also clarified before this that

      P-DAO 3 may be omitted if P-DAO 1 and 2 signal F and G as Targets.

> Would that impact the ability to deliver packets to F, G via (A, 129) ?

Not in the case of stitched if we do not have packets to E, but it is needed in other cases when the Leg to E signals F and G. This is why I chose to illustrate the case you propose, shows more stuff.

> If so, why ?  If its possible to deliver to (F,G) without indicating E in
> target list, hen i think the statement is wrong.
> 
> And i can't see how the logic works. Lets assume C has a neighbor H,
> so now we set the targets for P-DAO 2 to E,F,G,H. How would the notion
> of including C into this target list help C to decide wheher to deliver
> packets to H directly instead of leting them maybe go through further
> segments ? Aka: It seems to me that if the logic is that if you are
> sement endpoint, and any track egres for that segments P-DAO is a neighbor
> in the main dodag, then you do forward directly to that target.

C's routing table decides. Normally connected routes to neighbors win in the RIB vs routes from IGPs.
But C does not inject a RTO for H. The main Root selects what RTOs get in the P DAO

> 
> Yes/No/Maybe ? ;-))
> 
> Also: I think there should be some "Punt" line in the FIB on E for E as
> a target. And it seems to me that that Punt line would be created
> by including the segment endpoint into the target list. Maybe one
> does not want to be able to address a segment endpoint via the Track
> DODAG...
> 
>   1656	   Packets originated by A to E, F, or G, do not require an
>                              ^^^ from a source X via
> 
>   1657	   encapsulation.  In any fashion, the outer headers of the
> packets that
>            ^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^ remove
> 
> how about "any additional encapsulation beside the one outer encapsulation
> required for non-storage-mode" ?
> 
> Because immediately following you do write and show that outer
> encapsulation header.

You missed the originated by A piece. Clarified as follows:

   Packets originated by A to F or G do not require an encapsulation as
   the RPI can be placed in the native header chain.  For packets that
   it routes, A must encapsulate to add the RPI that signals the
   trackID; the outer headers of the packets that are forwarded along
   the Track have the following settings:

    +========+===================+===================+================+
    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID in RPI |
    +========+===================+===================+================+
    | Outer  |         A         |       F or G      |    (A, 129)    |
    +--------+-------------------+-------------------+----------------+
    | Inner  |       X != A      |       F or G      |      N/A       |
    +--------+-------------------+-------------------+----------------+


> 
>   1657	   encapsulation.  In any fashion, the outer headers of the
> packets that
>   1658	   are forwarded along the Track have the following settings:
>   1659
>   1660
> +========+===================+===================+================+
>   1661	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   1662
> +========+===================+===================+================+
>   1663	    | Outer  |         A         |     E, F or G     |    (A,
> 129)    |
>   1664	    +--------+-------------------+-------------------+---------
> -------+
>   1665	    | Inner  |       X != A      |     E, F or G     |      N/A
> |
>   1666	    +--------+-------------------+-------------------+---------
> -------+
> 
> Maybe add note: The Inner header is only required for X != A.

Which is what the text I added above says, we re in sync.

> 
>   1667
>   1668	                      Table 3: Packet header settings
>   1669
>   1670	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   1671
>   1672	   *  From P-DAO 2: A forwards to B and B forwards to C.
>   1673
>   1674	   *  From P-DAO 1: C forwards to D and D forwards to E.
>   1675
>   1676	   *  From Neighbor Cache Entry: C delivers the packet to F.
>                                          ^ E ?
> 
> But see above, your FIB shows that E -> F as part of (A, 129), whereas the
> neighbor
> cache entry is probably independent of 129, right ? So one of those
> statements
> (table or neighbor-cache-entry) is wrong.

That was not the meaning. The meaning was that it is signaled as target of that track.


> 
>   1685	9.1.2.  External routes
>                 ^^^^^^^^^^^^^^^
> 
> New term not used in before. Explain.

Added


   In this example, we consider F and G as destinations that are
   external to the Track as a DODAG, as discussed in section 4.1.1. of
   [RFC9008].  We then apply the directives for encapsulating in that
   case, more in Section 6.6.


> 
>   1687	   Storing-Mode P-DAO 1 and 2, and Non-Storing-Mode P-DAO 3,
> are sent to
>   1688	   E, C and A, respectively, as follows:
>   1689
>   1690
> +===============+==============+==============+==============+
>   1691	      |               | P-DAO 1 to E | P-DAO 2 to C | P-DAO 3
> to A |
>   1692
> +===============+==============+==============+==============+
>   1693	      |      Mode     | Storing      | Storing      | Non-
> Storing  |
>   1694	      +---------------+--------------+--------------+----------
> ----+
>   1695	      | Track Ingress | A            | A            | A
> |
>   1696	      +---------------+--------------+--------------+----------
> ----+
>   1697	      |    TrackID    | (A, 129)     | (A, 129)     | (A, 129)
> |
>   1698	      +---------------+--------------+--------------+----------
> ----+
>   1699	      |      VIO      | C, D, E      | A, B, C      | E
> |
>   1700	      +---------------+--------------+--------------+----------
> ----+
>   1701	      |    Targets    | E            | E            | F, G
> |
>   1702	      +---------------+--------------+--------------+----------
> ----+
> 
> Some explanation about the purpose of this example would be useful.
> 
> Seems to be something like. In this example, we want to avoid creating
> state for F,G on B,C,D. We do this by using the two storing mode segmnts
> P-DAO1 and P-DAO 2 to reach E and a non-storing segment P-DAO 3 to reach F
> and G.
> 

   In this formulation, we set up the Track Leg explicitly, which creates less
   routing state in intermediate hops at the expense of larger packets to
   accommodate source routing

> I am not clear why P-DAO 3 needs to be non-storing though. What would
> happen

Because the P DAO is passed hop by hop to the predecessor. Also it could create loops if done otherwise.
So we simplified and leave it at strict



> if the only change to above was to indicate P-DAO 3 as storing ? aka: why
> would B,C,D need to bother about any Target list of segments that they are
> not on ?
> 
>   1704	                         Table 4: P-DAO Messages
>   1705
>   1706	   As a result the RIBs are set as follows:
>   1707
>   1708
> +======+=============+=========+=============+==========+
>   1709	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   1710
> +======+=============+=========+=============+==========+
>   1711	         |  E   | F, G        | P-DAO 1 | Neighbor    | (A,
> 129) |
>                                         ^^^^^^^
> 
> Shouldn't this be P-DAO 3 ??
> 
>   1712	         +------+-------------+---------+-------------+--------
> --+
>   1713	         |  D   | E           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1714	         +------+-------------+---------+-------------+--------
> --+
>   1715	         |  C   | D           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1716	         +------+-------------+---------+-------------+--------
> --+
>   1717	         |  "   | E           | P-DAO 1 | D           | (A,
> 129) |
>   1718	         +------+-------------+---------+-------------+--------
> --+
>   1719	         |  B   | C           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1720	         +------+-------------+---------+-------------+--------
> --+
>   1721	         |  "   | E           | P-DAO 2 | C           | (A,
> 129) |
>   1722	         +------+-------------+---------+-------------+--------
> --+
>   1723	         |  A   | B           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1724	         +------+-------------+---------+-------------+--------
> --+
>   1725	         |  A   | E           | P-DAO 2 | B           | (A,
> 129) |
>   1726	         +------+-------------+---------+-------------+--------
> --+
>   1727	         |  A   | F, G        | P-DAO 3 | E           | (A,
> 129) |
>   1728	         +------+-------------+---------+-------------+--------
> --+
>   1729
>   1730	                            Table 5: RIB setting
>   1731
>   1741	   Packets from A to E do not require an encapsulation.  In any
> fashion,
>                                                  ^^^^^^^^^^^^^
> ^^^^^^^^^^^^^^  delete
>                                                  Inner Header
> 
>   1742	   the outer headers of the packets that are forwarded along
> the Track
>   1743	   have the following settings:
>   1744
>   1745
> +========+===================+====================+================+
>   1746	   | Header | IPv6 Source Addr. | IPv6 Dest.  Addr.  | TrackID
> in RPI |
>   1747
> +========+===================+====================+================+
>   1748	   | Outer  |         A         |         E          |    (A,
> 129)    |
>   1749	   +--------+-------------------+--------------------+---------
> -------+
>   1750	   | Inner  |         X         | E (X != A), F or G |      N/A
> |
>   1751	   +--------+-------------------+--------------------+---------
> -------+
>   1752
>   1753	                     Table 6: Packet header settings
>   1754
>   1755	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   1756
>   1757	   *  From P-DAO 3: A encapsulates the packet the Track
> signaled by
>                                                      ^ for     ^ (A,129)
>   1758	      P-DAO 3, with the outer header above.  Now the packet
> destination
>   1759	      is E.
>              ^ of the Outer Header
>   1760
>   1761	   *  From P-DAO 2: A forwards to B and B forwards to C.
>   1762
>   1763	   *  From P-DAO 1: C forwards to D and D forwards to E; E
> decapsulates
>   1764	      the packet.
>                         ^ because it is the destination of the Outer Header
>          ?? AND ? a valid target  for (A, 129) ??? (see question above
>   1765
>   1766	   *  From Neighbor Cache Entry: C delivers packets to F or G.
>   1767
>   1768	9.1.3.  Segment Routing
> 
> New term not used in before in this doc, define, reference (RFC8402 ?),
> etc. pp.


That was the intension. If you use P-DAO in ACP, you'll probably use the RFC 8754 as opposed to RFC 8138 for the SRH, won't you?

> 
>   1770	   Storing-Mode P-DAO 1 and 2, and Non-Storing-Mode P-DAO 3,
> are sent to
>   1771	   E, B and A, respectively, as follows:
> 
> Again: Please add purpose/goal of this example.

Yes, added to all. In this case that gives 


   *  P-DAO 1 signals C==>D==>E-to-E

   *  P-DAO 2 signals A==>B-to-B,C

   *  P-DAO 3 signals F and G via the A-->C-->E-to-F,G Track

> 
> If you did improve initial picture to show segments, then of course these
> ar different now.
> 
> 
>   1773
> +===============+==============+==============+==============+
>   1774	      |               | P-DAO 1 to E | P-DAO 2 to B | P-DAO 3
> to A |
>   1775
> +===============+==============+==============+==============+
>   1776	      |      Mode     | Storing      | Storing      | Non-
> Storing  |
>   1777	      +---------------+--------------+--------------+----------
> ----+
>   1778	      | Track Ingress | A            | A            | A
> |
>   1779	      +---------------+--------------+--------------+----------
> ----+
>   1780	      |    TrackID    | (A, 129)     | (A, 129)     | (A, 129)
> |
>   1781	      +---------------+--------------+--------------+----------
> ----+
>   1782	      |      VIO      | C, D, E      | A, B         | C, E
> |
>   1783	      +---------------+--------------+--------------+----------
> ----+
>   1784	      |    Targets    | E            | B, C         | F, G
> |
>   1785	      +---------------+--------------+--------------+----------
> ----+
>   1786
>   1787	                         Table 7: P-DAO Messages
> 
> Does this example only work with B being the exit node for P-DAO2, or could
> it equally work if we kept P-DAO 2 unchanged, ending at C, but the target
> only being C ?

Both work, I ack the depth of your review. You really went into it. Many thanks.

I added

   Note in the above that the Segment can terminate at the loose hop as
   used in the example of P-DAO 1 or at the previous hop as done with
   P-DAO 2.  Both methods are possible on any Segment joined by a loose
   Track Leg. P-DAO 1 generates more signaling since E is the Segment
   Egress when D could be, but has the benefit that it validates that
   the connectivity between D and E still exists.



> I am guessing it could be, given how C would also need to look into P-DAO1
> route
> towards E, right ? Aka: Minimal change over prior example might make it
> easier..

I like to illustrate both cases as opposed to mandate one



> 
>   1797	   As a result the RIBs are set as follows:
>   1798
>   1799
> +======+=============+=========+=============+==========+
>   1800	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   1801
> +======+=============+=========+=============+==========+
>   1802	         |  E   | F, G        | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1803	         +------+-------------+---------+-------------+--------
> --+
>   1804	         |  D   | E           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1805	         +------+-------------+---------+-------------+--------
> --+
>   1806	         |  C   | D           | P-DAO 1 | Neighbor    | (A,
> 129) |
>   1807	         +------+-------------+---------+-------------+--------
> --+
>   1808	         |  "   | E           | P-DAO 1 | D           | (A,
> 129) |
>   1809	         +------+-------------+---------+-------------+--------
> --+
>   1810	         |  B   | C           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1811	         +------+-------------+---------+-------------+--------
> --+
>   1812	         |  A   | B           | P-DAO 2 | Neighbor    | (A,
> 129) |
>   1813	         +------+-------------+---------+-------------+--------
> --+
>   1814	         |  "   | C           | P-DAO 2 | B           | (A,
> 129) |
>   1815	         +------+-------------+---------+-------------+--------
> --+
>   1816	         |  "   | E, F, G     | P-DAO 3 | C, E        | (A,
> 129) |
>   1817	         +------+-------------+---------+-------------+--------
> --+
>   1818
>   1819	                            Table 8: RIB setting
>   1820
>   1821	   Packets from A to E do not require an encapsulation, but
> carry a SRH
> 
> a third header instead of encap ?

Then again I mean originated by A.

> 
>   1822	   via C.  In any fashion, the outer headers of the packets
> that are
>   1823	   forwarded along the Track have the following settings:
>   1824
>   1825
> +========+===================+====================+================+
>   1826	   | Header | IPv6 Source Addr. | IPv6 Dest.  Addr.  | TrackID
> in RPI |
>                                           ^^^^^^^^^^^^^^^^^
> 
> the SRH for th Outer Header is not just an Pv6 Dest. Addr.
> 
>   1827
> +========+===================+====================+================+
>   1828	   | Outer  |         A         |  C till C then E   |    (A,
> 129)    |
>   1829	   +--------+-------------------+--------------------+---------
> -------+
>   1830	   | Inner  |         X         | E (X != A), F or G |      N/A
> |
>   1831	   +--------+-------------------+--------------------+---------
> -------+
>   1832
>   1833	                     Table 9: Packet header settings
>   1834
>   1835	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   1836
>   1837	   *  From P-DAO 3: A encapsulates the packet the Track
> signaled by
>   1838	      P-DAO 3, with the outer header above.  Now the
> destination in the
>   1839	      IPv6 Header is C, and a SRH signals the final destination
> is E.
>   1840
>   1841	   *  From P-DAO 2: A forwards to B and B forwards to C.
>   1842
>   1843	   *  From P-DAO 3: C processes the SRH and sets the
> destination in the
>   1844	      IPv6 Header to E.
>   1845
>   1853	   *  From P-DAO 1: C forwards to D and D forwards to E; E
> decapsulates
>   1854	      the packet.
>   1855
>   1856	   *  From the Neighbor Cache Entry: C delivers packets to F or
> G.
>   1857
>   1858	9.2.  Using Non-Storing-Mode joining Tracks
>   1859
>   1860	   A==>B==>C and C==>D==>E are Tracks expressed as non-storing
> P-DAOs.
>   1861
>   1862	9.2.1.  Stitched Tracks
> 
> sequential Tracks ?

Yes, that would work too.


> 
>   1864	   Non-Storing Mode P-DAO 1 and 2 are sent to C and A
> respectively, as
>   1865	   follows:
>   1866
>   1867	              +===============+==============+==============+
>   1868	              |               | P-DAO 1 to C | P-DAO 2 to A |
>   1869	              +===============+==============+==============+
>   1870	              |      Mode     | Non-Storing  | Non-Storing  |
>   1871	              +---------------+--------------+--------------+
>   1872	              | Track Ingress | C            | A            |
>   1873	              +---------------+--------------+--------------+
>   1874	              |    TrackID    | (C, 131)     | (A, 129)     |
>   1875	              +---------------+--------------+--------------+
>   1876	              |      VIO      | D, E         | B, C         |
>   1877	              +---------------+--------------+--------------+
>   1878	              |    Targets    | F, G         | E, F, G      |
>   1879	              +---------------+--------------+--------------+
> 
> WOuld it be possible for both DAO to have the same number (129) given how
> they are disambiguated by the source address ? If so i think it would
> strenthen
> the example by doing so and highlighting that these are not the same Tracks
> anymore,
> even if they reuse the same TrackID.

It would. I used TrackID of 131 in both cases and added

   A encapsulates the packet with destination of F in the Track signaled by
   P-DAO 2. The outer header has source A, destination B, an SRH that
   indicates C as the next loose hop, and a RPI indicating a TrackId of 131
   from A's namespace, which is distinct from TrackId of 131 from C's.
> 
>   1880
>   1881	                          Table 10: P-DAO Messages
>   1882
>   1883	   As a result the RIBs are set as follows:
>   1884
>   1885
>   1886
>   1887
>   1888
>   1889
>   1890
>   1891
>   1892
>   1893
>   1894
>   1895
>   1896
>   1897
>   1898
>   1899
>   1900
>   1901
>   1902
>   1903
>   1904	Thubert, et al.          Expires 28 January 2022
> [Page 34]
>   1905
> 
> 
>   1906	Internet-Draft               DAO Projection
> July 2021
>   1907
>   1908
>   1909
> +======+=============+=========+=============+==========+
>   1910	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   1911
> +======+=============+=========+=============+==========+
>   1912	         |  E   | F, G        | ND      | Neighbor    | Any
> |
>   1913	         +------+-------------+---------+-------------+--------
> --+
>   1914	         |  D   | E           | ND      | Neighbor    | Any
> |
>   1915	         +------+-------------+---------+-------------+--------
> --+
>   1916	         |  C   | D           | ND      | Neighbor    | Any
> |
>   1917	         +------+-------------+---------+-------------+--------
> --+
>   1918	         |  "   | E, F, G     | P-DAO 1 | D, E        | (C,
> 131) |
>   1919	         +------+-------------+---------+-------------+--------
> --+
>   1920	         |  B   | C           | ND      | Neighbor    | Any
> |
>   1921	         +------+-------------+---------+-------------+--------
> --+
>   1922	         |  A   | B           | ND      | Neighbor    | Any
> |
>   1923	         +------+-------------+---------+-------------+--------
> --+
>   1924	         |  "   | C, E, F, G  | P-DAO 2 | B, C        | (A,
> 129) |
>   1925	         +------+-------------+---------+-------------+--------
> --+
>   1926
>   1927	                           Table 11: RIB setting
>   1928
>   1929	   Packets from A to E, F and G do not require an
> encapsulation, though
>   1930	   it is preferred that A encapsulates and C decapsulates.
> Either way,
>   1931	   they carry a SRH via B and C, and C needs to encapsulate to
> E, F, or
>   1932	   G to add an SRH via D and E.  The encapsulating headers of
> packets
>   1933	   that are forwarded along the Track between C and E have the
> following
>   1934	   settings:
>   1935
>   1936
> +========+===================+===================+================+
>   1937	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   1938
> +========+===================+===================+================+
>   1939	    | Outer  |         C         |  D till D then E  |    (C,
> 131)    |
>   1940	    +--------+-------------------+-------------------+---------
> -------+
>   1941	    | Inner  |         X         |     E, F, or G    |      N/A
> |
>   1942	    +--------+-------------------+-------------------+---------
> -------+
>   1943
>   1944	                      Table 12: Packet header settings
>   1945
>   1946	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   1947
>   1948	   *  From P-DAO 2: A encapsulates the packet with destination
> of F in
>   1949	      the Track signaled by P-DAO 2.  The outer header has
> source A,
>   1950	      destination B, an SRH that indicates C as the next loose
> hop, and
>   1951	      a RPI indicating a TrackId of 129 from A's namespace.
>   1952
>   1953	   *  From the SRH: Packets forwarded by B have source A,
> destination C
>   1954	      , a consumed SRH, and a RPI indicating a TrackId of 129
> from A's
>   1955	      namespace.  C decapsulates.
>   1956
>   1965	   *  From P-DAO 1: C encapsulates the packet with destination
> of F in
>   1966	      the Track signaled by P-DAO 1.  The outer header has
> source C,
>   1967	      destination D, an SRH that indicates E as the next loose
> hop, and
>   1968	      a RPI indicating a TrackId of 131 from C's namespace.  E
>   1969	      decapsulates.
>   1970
>   1971	9.2.2.  External routes
> 
> sequential tracks with external routes ?
> 
>   1972
>   1973	   Non-Storing Mode P-DAO 1 is sent to C and Non-Storing Mode
> P-DAO 2
>   1974	   and 3 are sent A, as follows:
>   1975
>   1976
> +===============+==============+==============+==============+
>   1977	      |               | P-DAO 1 to C | P-DAO 2 to A | P-DAO 3
> to A |
>   1978
> +===============+==============+==============+==============+
>   1979	      |      Mode     | Non-Storing  | Non-Storing  | Non-
> Storing  |
>   1980	      +---------------+--------------+--------------+----------
> ----+
>   1981	      | Track Ingress | C            | A            | A
> |
>   1982	      +---------------+--------------+--------------+----------
> ----+
>   1983	      |    TrackID    | (C, 131)     | (A, 129)     | (A, 141)
> |
>   1984	      +---------------+--------------+--------------+----------
> ----+
>   1985	      |      VIO      | D, E         | B, C         | E
> |
>   1986	      +---------------+--------------+--------------+----------
> ----+
>   1987	      |    Targets    | E            | E            | F, G
> |
>   1988	      +---------------+--------------+--------------+----------
> ----+
>   1989
>   1990	                         Table 13: P-DAO Messages
>   1991
>   1992	   As a result the RIBs are set as follows:
>   1993
>   1994
>   1995
>   1996
>   1997
>   1998
>   1999
>   2000
>   2001
>   2002
>   2003
>   2004
>   2005
>   2006
>   2007
>   2008
>   2009
>   2010
>   2011
>   2012
>   2013
>   2014
>   2015
>   2016	Thubert, et al.          Expires 28 January 2022
> [Page 36]
>   2017
> 
> 
>   2018	Internet-Draft               DAO Projection
> July 2021
>   2019
>   2020
>   2021
> +======+=============+=========+=============+==========+
>   2022	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   2023
> +======+=============+=========+=============+==========+
>   2024	         |  E   | F, G        | ND      | Neighbor    | Any
> |
>   2025	         +------+-------------+---------+-------------+--------
> --+
>   2026	         |  D   | E           | ND      | Neighbor    | Any
> |
>   2027	         +------+-------------+---------+-------------+--------
> --+
>   2028	         |  C   | D           | ND      | Neighbor    | Any
> |
>   2029	         +------+-------------+---------+-------------+--------
> --+
>   2030	         |  "   | E           | P-DAO 1 | D, E        | (C,
> 131) |
>   2031	         +------+-------------+---------+-------------+--------
> --+
>   2032	         |  B   | C           | ND      | Neighbor    | Any
> |
>   2033	         +------+-------------+---------+-------------+--------
> --+
>   2034	         |  A   | B           | ND      | Neighbor    | Any
> |
>   2035	         +------+-------------+---------+-------------+--------
> --+
>   2036	         |  "   | C, E        | P-DAO 2 | B, C        | (A,
> 129) |
>   2037	         +------+-------------+---------+-------------+--------
> --+
>   2038	         |  "   | F, G        | P-DAO 3 | E           | (A,
> 141) |
>   2039	         +------+-------------+---------+-------------+--------
> --+
>   2040
>   2041	                           Table 14: RIB setting
>   2042
>   2043	   The encapsulating headers of packets that are forwarded
> along the
>   2044	   Track between C and E have the following settings:
>   2045
>   2046
> +========+===================+===================+================+
>   2047	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   2048
> +========+===================+===================+================+
>   2049	    | Outer  |         C         |  D till D then E  |    (C,
> 131)    |
>   2050	    +--------+-------------------+-------------------+---------
> -------+
>   2051	    | Middle |         A         |         E         |    (A,
> 141)    |
>   2052	    +--------+-------------------+-------------------+---------
> -------+
>   2053	    | Inner  |         X         |     E, F or G     |      N/A
> |
>   2054	    +--------+-------------------+-------------------+---------
> -------+
>   2055
>   2056	                      Table 15: Packet header settings
>   2057
>   2058	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   2059
>   2060	   *  From P-DAO 3: A encapsulates the packet with destination
> of F in
>   2061	      the Track signaled by P-DAO 3.  The outer header has
> source A,
>   2062	      destination E, and a RPI indicating a TrackId of 141 from
> A's
>   2063	      namespace.  This recurses with:
>   2064
>   2065	   *  From P-DAO 2: A encapsulates the packet with destination
> of E in
>   2066	      the Track signaled by P-DAO 2.  The outer header has
> source A,
>   2067	      destination B, an SRH that indicates C as the next loose
> hop, and
>   2068	      a RPI indicating a TrackId of 129 from A's namespace.
>   2069
>   2070
>   2071
>   2072	Thubert, et al.          Expires 28 January 2022
> [Page 37]
>   2073
> 
> 
>   2074	Internet-Draft               DAO Projection
> July 2021
>   2075
>   2076
>   2077	   *  From the SRH: Packets forwarded by B have source A,
> destination C
>   2078	      , a consumed SRH, and a RPI indicating a TrackId of 129
> from A's
>   2079	      namespace.  C decapsulates.
>   2080
>   2081	   *  From P-DAO 1: C encapsulates the packet with destination
> of E in
>   2082	      the Track signaled by P-DAO 1.  The outer header has
> source C,
>   2083	      destination D, an SRH that indicates E as the next loose
> hop, and
>   2084	      a RPI indicating a TrackId of 131 from C's namespace.  E
>   2085	      decapsulates.
>   2086
>   2087	9.2.3.  Segment Routing
> 
> segment routing with exernal routes ?
>   2088
>   2089	   Non-Storing Mode P-DAO 1 is sent to C and Non-Storing Mode
> P-DAO 2
>   2090	   and 3 are sent A, as follows:
>   2091
>   2092
> +===============+==============+==============+==============+
>   2093	      |               | P-DAO 1 to C | P-DAO 2 to A | P-DAO 3
> to A |
>   2094
> +===============+==============+==============+==============+
>   2095	      |      Mode     | Non-Storing  | Non-Storing  | Non-
> Storing  |
>   2096	      +---------------+--------------+--------------+----------
> ----+
>   2097	      | Track Ingress | C            | A            | A
> |
>   2098	      +---------------+--------------+--------------+----------
> ----+
>   2099	      |    TrackID    | (C, 131)     | (A, 129)     | (A, 141)
> |
>   2100	      +---------------+--------------+--------------+----------
> ----+
>   2101	      |      VIO      | D, E         | B            | C, E
> |
>   2102	      +---------------+--------------+--------------+----------
> ----+
>   2103	      |    Targets    |              | C            | F, G
> |
>   2104	      +---------------+--------------+--------------+----------
> ----+
>   2105
>   2106	                         Table 16: P-DAO Messages
>   2107
>   2108	   As a result the RIBs are set as follows:
>   2109
>   2110
>   2111
>   2112
>   2113
>   2114
>   2115
>   2116
>   2117
>   2118
>   2119
>   2120
>   2121
>   2122
>   2123
>   2124
>   2125
>   2126
>   2127
>   2128	Thubert, et al.          Expires 28 January 2022
> [Page 38]
>   2129
> 
> 
>   2130	Internet-Draft               DAO Projection
> July 2021
>   2131
>   2132
>   2133
> +======+=============+=========+=============+==========+
>   2134	         | Node | Destination | Origin  | Next Hop(s) | TrackID
> |
>   2135
> +======+=============+=========+=============+==========+
>   2136	         |  E   | F, G        | ND      | Neighbor    | Any
> |
>   2137	         +------+-------------+---------+-------------+--------
> --+
>   2138	         |  D   | E           | ND      | Neighbor    | Any
> |
>   2139	         +------+-------------+---------+-------------+--------
> --+
>   2140	         |  C   | D           | ND      | Neighbor    | Any
> |
>   2141	         +------+-------------+---------+-------------+--------
> --+
>   2142	         |  "   | E           | P-DAO 1 | D, E        | (C,
> 131) |
>   2143	         +------+-------------+---------+-------------+--------
> --+
>   2144	         |  B   | C           | ND      | Neighbor    | Any
> |
>   2145	         +------+-------------+---------+-------------+--------
> --+
>   2146	         |  A   | B           | ND      | Neighbor    | Any
> |
>   2147	         +------+-------------+---------+-------------+--------
> --+
>   2148	         |  "   | C           | P-DAO 2 | B, C        | (A,
> 129) |
>   2149	         +------+-------------+---------+-------------+--------
> --+
>   2150	         |  "   | E, F, G     | P-DAO 3 | C, E        | (A,
> 141) |
>   2151	         +------+-------------+---------+-------------+--------
> --+
>   2152
>   2153	                           Table 17: RIB setting
>   2154
>   2155	   The encapsulating headers of packets that are forwarded
> along the
>   2156	   Track between A and B have the following settings:
>   2157
>   2158
> +========+===================+===================+================+
>   2159	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   2160
> +========+===================+===================+================+
>   2161	    | Outer  |         A         |  B till D then E  |    (A,
> 129)    |
>   2162	    +--------+-------------------+-------------------+---------
> -------+
>   2163	    | Middle |         A         |         C         |    (A,
> 141)    |
>   2164	    +--------+-------------------+-------------------+---------
> -------+
>   2165	    | Inner  |         X         |     E, F or G     |      N/A
> |
>   2166	    +--------+-------------------+-------------------+---------
> -------+
>   2167
>   2168	                      Table 18: Packet header settings
>   2169
>   2170	   The encapsulating headers of packets that are forwarded
> along the
>   2171	   Track between B and C have the following settings:
>   2172
>   2173
>   2174
>   2175
>   2176
>   2177
>   2178
>   2179
>   2180
>   2181
>   2182
>   2183
>   2184	Thubert, et al.          Expires 28 January 2022
> [Page 39]
>   2185
> 
> 
>   2186	Internet-Draft               DAO Projection
> July 2021
>   2187
>   2188
>   2189
> +========+===================+===================+================+
>   2190	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   2191
> +========+===================+===================+================+
>   2192	    | Outer  |         A         |         C         |    (A,
> 141)    |
>   2193	    +--------+-------------------+-------------------+---------
> -------+
>   2194	    | Inner  |         X         |     E, F or G     |      N/A
> |
>   2195	    +--------+-------------------+-------------------+---------
> -------+
>   2196
>   2197	                      Table 19: Packet header settings
>   2198
>   2199	   The encapsulating headers of packets that are forwarded
> along the
>   2200	   Track between C and E have the following settings:
>   2201
>   2202
> +========+===================+===================+================+
>   2203	    | Header | IPv6 Source Addr. | IPv6 Dest.  Addr. | TrackID
> in RPI |
>   2204
> +========+===================+===================+================+
>   2205	    | Outer  |         C         |  D till D then E  |    (C,
> 131)    |
>   2206	    +--------+-------------------+-------------------+---------
> -------+
>   2207	    | Middle |         A         |         E         |    (A,
> 141)    |
>   2208	    +--------+-------------------+-------------------+---------
> -------+
>   2209	    | Inner  |         X         |     E, F or G     |      N/A
> |
>   2210	    +--------+-------------------+-------------------+---------
> -------+
>   2211
>   2212	                      Table 20: Packet header settings
>   2213
>   2214	   As an example, say that A has a packet for F.  Using the RIB
> above:
>   2215
>   2216	   *  From P-DAO 3: A encapsulates the packet with destination
> of F in
>   2217	      the Track signaled by P-DAO 3.  The outer header has
> source A,
>   2218	      destination C, an SRH that indicates E as the next loose
> hop, and
>   2219	      a RPI indicating a TrackId of 141 from A's namespace.
> This
>   2220	      recurses with:
>   2221
>   2222	   *  From P-DAO 2: A encapsulates the packet with destination
> of C in
>   2223	      the Track signaled by P-DAO 2.  The outer header has
> source A,
>   2224	      destination B, and a RPI indicating a TrackId of 129 from
> A's
>   2225	      namespace.  B decapsulates forwards to C based on a
> sibling
>   2226	      connected route.
>   2227
>   2228	   *  From the SRH: C consumes the SRH and makes the
> destination E.
>   2229
>   2230	   *  From P-DAO 1: C encapsulates the packet with destination
> of E in
>   2231	      the Track signaled by P-DAO 1.  The outer header has
> source C,
>   2232	      destination D, an SRH that indicates E as the next loose
> hop, and
>   2233	      a RPI indicating a TrackId of 131 from C's namespace.  E
>   2234	      decapsulates.
>   2235
> 
> Sorry, i skipped mostly across 1883 to here, running out of time.
> 
>   2245	10.  Security Considerations
>   2246
>   2247	   It is worth noting that with [RPL], every node in the LLN is
> RPL-
>   2248	   aware and can inject any RPL-based attack in the network.
> This draft
> 
> I guss even RUL nodes, which i think came after rfc6550 could be nasty too
> and
> inject RPL messages.

But they should be filtered, that's the whole point of forcing them to speak RFC 8505 only.

> 
>   2249	   uses messages that are already present in RPL [RPL] with
> optional
>   2250	   secured versions.  The same secured versions may be used
> with this
>   2251	   draft, and whatever security is deployed for a given network
> also
>   2252	   applies to the flows in this draft.
>   2253
>   2254	   The LLN nodes depend on the 6LBR and the RPL participants
> for their
>   2255	   operation.  A trust model must be put in place to ensure
> that the
>   2256	   right devices are acting in these roles, so as to avoid
> threats such
>   2257	   as black-holing, (see [RFC7416] section 7).  This trust
> model could
>   2258	   be at a minimum based on a Layer-2 Secure joining and the
> Link-Layer
>   2259	   security.  This is a generic 6LoWPAN requirement, see Req5.1
> in
>   2260	   Appendix B.5 of [RFC8505].
> 
>   2261
>   2262	   In a general manner, the Security Considerations in [RPL],
> and
>   2263	   [RFC7416] apply to this specification as well.  The Link-
> Layer
>   2264	   security is needed in particular to prevent Denial-Of-
> Service attacks
>   2265	   whereby a rogue router creates a high churn in the RPL
> network by
>   2266	   constantly injected forged P-DAO messages and using up all
> the
>   2267	   available storage in the attacked routers.
> 
> Seems like the answer is no.
> 
>   2268
>   2269	   Additionally, the trust model could include a role
> validation (e.g.,
>   2270	   using a role-based authorization) to ensure that the node
> that claims
>   2271	   to be a RPL Root is entitled to do so.  That trust should
> propagate
>   2272	   from egress to ingress in the case of a Storing Mode P-DAO.
> 
> In ANIMA i am pondering to get role-based certificates to enable something
> like this.
> Is there any existing solution for this ? 

Not that I know of, let's see what Adrian's interim leads to

> If not, then it would be good to mention this
> as a gap. In general, the whole PCE based routing model does put more
> reliance of a
> trustworthy PCE into the solution, but it eliminates  a lot of security
> attacks
> coming from the normal peer-2-peer routing model (i am just making this
> argument
> in my bier-te draft, ask me again, when i have read BenK's feedback ;-).

😊

> 
> But when
> like it seems in this solution, one mixes the tradtional non-role based
> signaling
> with PCE input then it looks like a bigge gap than in before, because now
> these
> new routes allow to potentially create even more black holes than may have
> been
> possible in before (not quite sure).
> 
> In any case, you can also mention that you did try to eliminate one new
> type of
> attack by injection of duplicate addresses in tracks by mandating to
> discover such
> issue. Not sure though if this is the best set of heuristics that nodes
> could
> apply to self-defend themslfes and further hops from incorrect new
> messages. Miht
> be worth to ponder.


Added

    This specification suggests some validation of the VIO to prevent basic
    loops by avoiding that a node appears twice. But that is only a minimal
    protection. Arguably, an attacker tha can inject P-DAOs can reroute any
    traffic and deplete critical resources such as spectrum and battery in
    the LLN rapidly.

> 
>   2273
>   2274	11.  IANA Considerations
> 
> I am not going to read this (out of time). I would suggest as mentione din
> before to replace
> all numbers with TBDi and then strongly consider early IANA allocation.
> That would
> also help a lot to resolve any possible issues with any of the registration
> asks by
> mean of the IANA/expert review ensueing.

Added (Suggested)

> 
>   2276	11.1.  New Elective 6LoWPAN Routing Header Type
>   2277
>   2278	   This document updates the IANA registry titled "Elective
> 6LoWPAN
>   2279	   Routing Header Type" that was created for [RFC8138] and
> assigns the
>   2280	   following value:
>   2281
>   2282	                  +=======+=============+===============+
>   2283	                  | Value | Description | Reference     |
>   2284	                  +=======+=============+===============+
>   2285	                  |   7   | P-RPI-6LoRH | This document |
>   2286	                  +-------+-------------+---------------+
>   2287
>   2288	                       Table 21: New Elective 6LoWPAN
>   2289	                            Routing Header Type
>   2290
>   2291
>   2292
>   2293
>   2294
>   2295
>   2296	Thubert, et al.          Expires 28 January 2022
> [Page 41]
>   2297
> 
> 
>   2298	Internet-Draft               DAO Projection
> July 2021
>   2299
>   2300
>   2301	11.2.  New Critical 6LoWPAN Routing Header Type
>   2302
>   2303	   This document updates the IANA registry titled "Critical
> 6LoWPAN
>   2304	   Routing Header Type" that was created for [RFC8138] and
> assigns the
>   2305	   following value:
>   2306
>   2307	                  +=======+=============+===============+
>   2308	                  | Value | Description | Reference     |
>   2309	                  +=======+=============+===============+
>   2310	                  |   7   | P-RPI-6LoRH | This document |
>   2311	                  +-------+-------------+---------------+
>   2312
>   2313	                       Table 22: New Critical 6LoWPAN
>   2314	                            Routing Header Type
>   2315
>   2316	11.3.  New Subregistry For The RPL Option Flags
>   2317
>   2318	   IANA is required to create a subregistry for the 8-bit RPL
> Option
>   2319	   Flags field, as detailed in Figure 2, under the "Routing
> Protocol for
>   2320	   Low Power and Lossy Networks (RPL)" registry.  The bits are
> indexed
>   2321	   from 0 (leftmost) to 7.  Each bit is tracked with the
> following
>   2322	   qualities:
>   2323
>   2324	   *  Bit number (counting from bit 0 as the most significant
> bit)
>   2325
>   2326	   *  Indication When Set
>   2327
>   2328	   *  Reference
>   2329
>   2330	   Registration procedure is "Standards Action" [RFC8126].  The
> initial
>   2331	   allocation is as indicated in Table 26:
>   2332
>   2333
> +============+======================+===============+
>   2334	           | Bit number | Indication When Set  | Reference
> |
>   2335
> +============+======================+===============+
>   2336	           |     0      | Down 'O'             | [RFC6553]
> |
>   2337	           +------------+----------------------+---------------
> +
>   2338	           |     1      | Rank-Error (R)       | [RFC6553]
> |
>   2339	           +------------+----------------------+---------------
> +
>   2340	           |     2      | Forwarding-Error (F) | [RFC6553]
> |
>   2341	           +------------+----------------------+---------------
> +
>   2342	           |     3      | Projected-Route (P)  | This document
> |
>   2343	           +------------+----------------------+---------------
> +
>   2344
>   2345	                        Table 23: Initial PDR Flags
>   2346
>   2347
>   2348
>   2349
>   2350
>   2351
>   2352	Thubert, et al.          Expires 28 January 2022
> [Page 42]
>   2353
> 
> 
>   2354	Internet-Draft               DAO Projection
> July 2021
>   2355
>   2356
>   2357	11.4.  New RPL Control Codes
>   2358
>   2359	   This document extends the IANA Subregistry created by RFC
> 6550 for
>   2360	   RPL Control Codes as indicated in Table 24:
>   2361
>   2362
> +======+=============================+===============+
>   2363	          | Code | Description                 | Reference
> |
>   2364
> +======+=============================+===============+
>   2365	          | 0x09 | Projected DAO Request (PDR) | This document
> |
>   2366	          +------+-----------------------------+---------------
> +
>   2367	          | 0x0A | PDR-ACK                     | This document
> |
>   2368	          +------+-----------------------------+---------------
> +
>   2369
>   2370	                     Table 24: New RPL Control Codes
>   2371
>   2372	11.5.  New RPL Control Message Options
>   2373
>   2374	   This document extends the IANA Subregistry created by RFC
> 6550 for
>   2375	   RPL Control Message Options as indicated in Table 25:
>   2376
>   2377
> +=======+============================+===============+
>   2378	          | Value | Meaning                    | Reference
> |
>   2379
> +=======+============================+===============+
>   2380	          |  0x0B | Stateful VIO (SF-VIO)      | This document
> |
>   2381	          +-------+----------------------------+---------------
> +
>   2382	          |  0x0C | Source-Routed VIO (SR-VIO) | This document
> |
>   2383	          +-------+----------------------------+---------------
> +
>   2384	          |  0x0D | Sibling Information option | This document
> |
>   2385	          +-------+----------------------------+---------------
> +
>   2386
>   2387	                  Table 25: RPL Control Message Options
>   2388
>   2389	11.6.  SubRegistry for the Projected DAO Request Flags
>   2390
>   2391	   IANA is required to create a registry for the 8-bit
> Projected DAO
>   2392	   Request (PDR) Flags field.  Each bit is tracked with the
> following
>   2393	   qualities:
>   2394
>   2395	   *  Bit number (counting from bit 0 as the most significant
> bit)
>   2396
>   2397	   *  Capability description
>   2398
>   2399	   *  Reference
>   2400
>   2401	   Registration procedure is "Standards Action" [RFC8126].  The
> initial
>   2402	   allocation is as indicated in Table 26:
>   2403
>   2404
>   2405
>   2406
>   2407
>   2408	Thubert, et al.          Expires 28 January 2022
> [Page 43]
>   2409
> 
> 
>   2410	Internet-Draft               DAO Projection
> July 2021
>   2411
>   2412
>   2413
> +============+========================+===============+
>   2414	          | Bit number | Capability description | Reference
> |
>   2415
> +============+========================+===============+
>   2416	          |     0      | PDR-ACK request (K)    | This document
> |
>   2417	          +------------+------------------------+--------------
> -+
>   2418	          |     1      | Requested path should  | This document
> |
>   2419	          |            | be redundant (R)       |
> |
>   2420	          +------------+------------------------+--------------
> -+
>   2421
>   2422	                        Table 26: Initial PDR Flags
>   2423
>   2424	11.7.  SubRegistry for the PDR-ACK Flags
>   2425
>   2426	   IANA is required to create an subregistry for the 8-bit PDR-
> ACK Flags
>   2427	   field.  Each bit is tracked with the following qualities:
>   2428
>   2429	   *  Bit number (counting from bit 0 as the most significant
> bit)
>   2430
>   2431	   *  Capability description
>   2432
>   2433	   *  Reference
>   2434
>   2435	   Registration procedure is "Standards Action" [RFC8126].  No
> bit is
>   2436	   currently defined for the PDR-ACK Flags.
>   2437
>   2438	11.8.  Subregistry for the PDR-ACK Acceptance Status Values
>   2439
>   2440	   IANA is requested to create a Subregistry for the PDR-ACK
> Acceptance
>   2441	   Status values.
>   2442
>   2443	   *  Possible values are 6-bit unsigned integers (0..63).
>   2444
>   2445	   *  Registration procedure is "Standards Action" [RFC8126].
>   2446
>   2447	   *  Initial allocation is as indicated in Table 27:
>   2448
>   2449	            +-------+------------------------+---------------+
>   2450	            | Value | Meaning                | Reference     |
>   2451	            +-------+------------------------+---------------+
>   2452	            | 0     | Unqualified acceptance | This document |
>   2453	            +-------+------------------------+---------------+
>   2454
>   2455	            Table 27: Acceptance values of the PDR-ACK Status
>   2456
>   2457	11.9.  Subregistry for the PDR-ACK Rejection Status Values
>   2458
>   2459	   IANA is requested to create a Subregistry for the PDR-ACK
> Rejection
>   2460	   Status values.
>   2461
>   2462
>   2463
>   2464	Thubert, et al.          Expires 28 January 2022
> [Page 44]
>   2465
> 
> 
>   2466	Internet-Draft               DAO Projection
> July 2021
>   2467
>   2468
>   2469	   *  Possible values are 6-bit unsigned integers (0..63).
>   2470
>   2471	   *  Registration procedure is "Standards Action" [RFC8126].
>   2472
>   2473	   *  Initial allocation is as indicated in Table 28:
>   2474
>   2475	             +-------+-----------------------+---------------+
>   2476	             | Value | Meaning               | Reference     |
>   2477	             +-------+-----------------------+---------------+
>   2478	             | 0     | Unqualified rejection | This document |
>   2479	             +-------+-----------------------+---------------+
>   2480
>   2481	              Table 28: Rejection values of the PDR-ACK Status
>   2482
>   2483	11.10.  SubRegistry for the Via Information Options Flags
>   2484
>   2485	   IANA is requested to create a Subregistry for the 5-bit Via
>   2486	   Information Options (Via Information Option) Flags field.
> Each bit
>   2487	   is tracked with the following qualities:
>   2488
>   2489	   *  Bit number (counting from bit 0 as the most significant
> bit)
>   2490
>   2491	   *  Capability description
>   2492
>   2493	   *  Reference
>   2494
>   2495	   Registration procedure is "Standards Action" [RFC8126].  No
> bit is
>   2496	   currently defined for the Via Information Options (Via
> Information
>   2497	   Option) Flags.
>   2498
>   2499	11.11.  SubRegistry for the Sibling Information Option Flags
>   2500
>   2501	   IANA is required to create a registry for the 5-bit Sibling
>   2502	   Information Option (SIO) Flags field.  Each bit is tracked
> with the
>   2503	   following qualities:
>   2504
>   2505	   *  Bit number (counting from bit 0 as the most significant
> bit)
>   2506
>   2507	   *  Capability description
>   2508
>   2509	   *  Reference
>   2510
>   2511	   Registration procedure is "Standards Action" [RFC8126].  The
> initial
>   2512	   allocation is as indicated in Table 29:
>   2513
>   2514
>   2515
>   2516
>   2517
>   2518
>   2519
>   2520	Thubert, et al.          Expires 28 January 2022
> [Page 45]
>   2521
> 
> 
>   2522	Internet-Draft               DAO Projection
> July 2021
>   2523
>   2524
>   2525
> +============+===================================+===============+
>   2526	    | Bit number | Capability description            |
> Reference     |
>   2527
> +============+===================================+===============+
>   2528	    |     0      | Connectivity is bidirectional (B) | This
> document |
>   2529	    +------------+-----------------------------------+---------
> ------+
>   2530
>   2531	                       Table 29: Initial SIO Flags
>   2532
>   2533	11.12.  New Destination Advertisement Object Flag
>   2534
>   2535	   This document modifies the "Destination Advertisement Object
> (DAO)
>   2536	   Flags" registry initially created in Section 20.11 of [RPL]
> .
>   2537
>   2538	   Section 3.1 also defines one new entry in the Registry as
> follows:
>   2539
>   2540	          +---------------+------------------------+-----------
> +
>   2541	          | Bit Number    | Capability Description | Reference
> |
>   2542	          +---------------+------------------------+-----------
> +
>   2543	          | 2 (suggested) | Projected DAO (P)      | THIS RFC
> |
>   2544	          +---------------+------------------------+-----------
> +
>   2545
>   2546	              Table 30: New Destination Advertisement Object
>   2547	                                (DAO) Flag
>   2548
>   2549	11.13.  Error in Projected Route ICMPv6 Code
>   2550
>   2551	   In some cases RPL will return an ICMPv6 error message when a
> message
>   2552	   cannot be forwarded along a Projected Route.  This ICMPv6
> error
>   2553	   message is "Error in Projected Route".
>   2554
>   2555	   IANA has defined an ICMPv6 "Code" Fields Registry for ICMPv6
> Message
>   2556	   Types.  ICMPv6 Message Type 1 describes "Destination
> Unreachable"
>   2557	   codes.  This specification requires that a new code is
> allocated from
>   2558	   the ICMPv6 Code Fields Registry for ICMPv6 Message Type 1,
> for "Error
>   2559	   in Projected Route", with a suggested code value of 8, to be
>   2560	   confirmed by IANA.
>   2561
>   2562	12.  Acknowledgments
>   2563
>   2564	   The authors wish to acknowledge JP Vasseur, Remy Liubing,
> James
>   2565	   Pylakutty and Patrick Wetterwald for their contributions to
> the ideas
>   2566	   developed here.
>   2567
>   2568	13.  Normative References
>   2569
>   2570
>   2571
>   2572
>   2573
>   2574
>   2575
>   2576	Thubert, et al.          Expires 28 January 2022
> [Page 46]
>   2577
> 
> 
>   2578	Internet-Draft               DAO Projection
> July 2021
>   2579
>   2580
>   2581	   [RFC2119]  Bradner, S., "Key words for use in RFCs to
> Indicate
>   2582	              Requirement Levels", BCP 14, RFC 2119,
>   2583	              DOI 10.17487/RFC2119, March 1997,
>   2584	              <https://www.rfc-editor.org/info/rfc2119>.
>   2585
>   2586	   [RFC4443]  Conta, A., Deering, S., and M. Gupta, Ed.,
> "Internet
>   2587	              Control Message Protocol (ICMPv6) for the
> Internet
>   2588	              Protocol Version 6 (IPv6) Specification", STD 89,
>   2589	              RFC 4443, DOI 10.17487/RFC4443, March 2006,
>   2590	              <https://www.rfc-editor.org/info/rfc4443>.
>   2591
>   2592	   [RFC6282]  Hui, J., Ed. and P. Thubert, "Compression Format
> for IPv6
>   2593	              Datagrams over IEEE 802.15.4-Based Networks", RFC
> 6282,
>   2594	              DOI 10.17487/RFC6282, September 2011,
>   2595	              <https://www.rfc-editor.org/info/rfc6282>.
>   2596
>   2597	   [RPL]      Winter, T., Ed., Thubert, P., Ed., Brandt, A.,
> Hui, J.,
>   2598	              Kelsey, R., Levis, P., Pister, K., Struik, R.,
> Vasseur,
>   2599	              JP., and R. Alexander, "RPL: IPv6 Routing
> Protocol for
>   2600	              Low-Power and Lossy Networks", RFC 6550,
>   2601	              DOI 10.17487/RFC6550, March 2012,
>   2602	              <https://www.rfc-editor.org/info/rfc6550>.
>   2603
>   2604	   [RFC6553]  Hui, J. and JP. Vasseur, "The Routing Protocol
> for Low-
>   2605	              Power and Lossy Networks (RPL) Option for
> Carrying RPL
>   2606	              Information in Data-Plane Datagrams", RFC 6553,
>   2607	              DOI 10.17487/RFC6553, March 2012,
>   2608	              <https://www.rfc-editor.org/info/rfc6553>.
>   2609
>   2610	   [RFC6554]  Hui, J., Vasseur, JP., Culler, D., and V. Manral,
> "An IPv6
>   2611	              Routing Header for Source Routes with the Routing
> Protocol
>   2612	              for Low-Power and Lossy Networks (RPL)", RFC
> 6554,
>   2613	              DOI 10.17487/RFC6554, March 2012,
>   2614	              <https://www.rfc-editor.org/info/rfc6554>.
>   2615
>   2616	   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase
> in RFC
>   2617	              2119 Key Words", BCP 14, RFC 8174, DOI
> 10.17487/RFC8174,
>   2618	              May 2017, <https://www.rfc-
> editor.org/info/rfc8174>.
>   2619
>   2620	   [RFC8126]  Cotton, M., Leiba, B., and T. Narten, "Guidelines
> for
>   2621	              Writing an IANA Considerations Section in RFCs",
> BCP 26,
>   2622	              RFC 8126, DOI 10.17487/RFC8126, June 2017,
>   2623	              <https://www.rfc-editor.org/info/rfc8126>.
>   2624
>   2625	14.  Informative References
>   2626
>   2627
>   2628
>   2629
>   2630
>   2631
>   2632	Thubert, et al.          Expires 28 January 2022
> [Page 47]
>   2633
> 
> 
>   2634	Internet-Draft               DAO Projection
> July 2021
>   2635
>   2636
>   2637	   [RFC7102]  Vasseur, JP., "Terms Used in Routing for Low-
> Power and
>   2638	              Lossy Networks", RFC 7102, DOI 10.17487/RFC7102,
> January
>   2639	              2014, <https://www.rfc-editor.org/info/rfc7102>.
>   2640
>   2641	   [RFC6997]  Goyal, M., Ed., Baccelli, E., Philipp, M.,
> Brandt, A., and
>   2642	              J. Martocci, "Reactive Discovery of Point-to-
> Point Routes
>   2643	              in Low-Power and Lossy Networks", RFC 6997,
>   2644	              DOI 10.17487/RFC6997, August 2013,
>   2645	              <https://www.rfc-editor.org/info/rfc6997>.
>   2646
>   2647	   [RFC7416]  Tsao, T., Alexander, R., Dohler, M., Daza, V.,
> Lozano, A.,
>   2648	              and M. Richardson, Ed., "A Security Threat
> Analysis for
>   2649	              the Routing Protocol for Low-Power and Lossy
> Networks
>   2650	              (RPLs)", RFC 7416, DOI 10.17487/RFC7416, January
> 2015,
>   2651	              <https://www.rfc-editor.org/info/rfc7416>.
>   2652
>   2653	   [6TiSCH-ARCHI]
>   2654	              Thubert, P., Ed., "An Architecture for IPv6 over
> the Time-
>   2655	              Slotted Channel Hopping Mode of IEEE 802.15.4
> (6TiSCH)",
>   2656	              RFC 9030, DOI 10.17487/RFC9030, May 2021,
>   2657	              <https://www.rfc-editor.org/info/rfc9030>.
>   2658
>   2659	   [RAW-ARCHI]
>   2660	              Thubert, P., Papadopoulos, G. Z., and L. Berger,
> "Reliable
>   2661	              and Available Wireless Architecture/Framework",
> Work in
>   2662	              Progress, Internet-Draft, draft-ietf-raw-
> architecture-00,
>   2663	              12 July 2021,
> <https://datatracker.ietf.org/doc/html/
>   2664	              draft-ietf-raw-architecture-00>.
>   2665
>   2666	   [TURN-ON_RFC8138]
>   2667	              Thubert, P., Ed. and L. Zhao, "A Routing Protocol
> for Low-
>   2668	              Power and Lossy Networks (RPL) Destination-
> Oriented
>   2669	              Directed Acyclic Graph (DODAG) Configuration
> Option for
>   2670	              the 6LoWPAN Routing Header", RFC 9035,
>   2671	              DOI 10.17487/RFC9035, April 2021,
>   2672	              <https://www.rfc-editor.org/info/rfc9035>.
>   2673
>   2674	   [RFC8655]  Finn, N., Thubert, P., Varga, B., and J. Farkas,
>   2675	              "Deterministic Networking Architecture", RFC
> 8655,
>   2676	              DOI 10.17487/RFC8655, October 2019,
>   2677	              <https://www.rfc-editor.org/info/rfc8655>.
>   2678
>   2679	   [RFC8025]  Thubert, P., Ed. and R. Cragie, "IPv6 over Low-
> Power
>   2680	              Wireless Personal Area Network (6LoWPAN) Paging
> Dispatch",
>   2681	              RFC 8025, DOI 10.17487/RFC8025, November 2016,
>   2682	              <https://www.rfc-editor.org/info/rfc8025>.
>   2683
>   2684
>   2685
>   2686
>   2687
>   2688	Thubert, et al.          Expires 28 January 2022
> [Page 48]
>   2689
> 
> 
>   2690	Internet-Draft               DAO Projection
> July 2021
>   2691
>   2692
>   2693	   [RFC8138]  Thubert, P., Ed., Bormann, C., Toutain, L., and
> R. Cragie,
>   2694	              "IPv6 over Low-Power Wireless Personal Area
> Network
>   2695	              (6LoWPAN) Routing Header", RFC 8138, DOI
> 10.17487/RFC8138,
>   2696	              April 2017, <https://www.rfc-
> editor.org/info/rfc8138>.
>   2697
>   2698	   [RFC8505]  Thubert, P., Ed., Nordmark, E., Chakrabarti, S.,
> and C.
>   2699	              Perkins, "Registration Extensions for IPv6 over
> Low-Power
>   2700	              Wireless Personal Area Network (6LoWPAN) Neighbor
>   2701	              Discovery", RFC 8505, DOI 10.17487/RFC8505,
> November 2018,
>   2702	              <https://www.rfc-editor.org/info/rfc8505>.
>   2703
>   2704	   [USEofRPLinfo]
>   2705	              Robles, M.I., Richardson, M., and P. Thubert,
> "Using RPI
>   2706	              Option Type, Routing Header for Source Routes,
> and IPv6-
>   2707	              in-IPv6 Encapsulation in the RPL Data Plane", RFC
> 9008,
>   2708	              DOI 10.17487/RFC9008, April 2021,
>   2709	              <https://www.rfc-editor.org/info/rfc9008>.
>   2710
>   2711	   [PCE]      IETF, "Path Computation Element",
>   2712	              <https://datatracker.ietf.org/doc/charter-ietf-
> pce/>.
>   2713
>   2714	Appendix A.  Applications
> 
> I wouldn't call this applications, its more like optimiations goals or the
> like
>   2715
>   2716	A.1.  Loose Source Routing
> 
> Optimizing packet size vs. routing state via Loose Source Routing
> 
>   2718	   A RPL implementation operating in a very constrained LLN
> typically
>   2719	   uses the Non-Storing Mode of Operation as represented in
> Figure 12.
>   2720	   In that mode, a RPL node indicates a parent-child
> relationship to the
>   2721	   Root, using a Destination Advertisement Object (DAO) that is
> unicast
>   2722	   from the node directly to the Root, and the Root typically
> builds a
>   2723	   source routed path to a destination down the DODAG by
> recursively
>   2724	   concatenating this information.
>   2725
>   2726	              ------+---------
>   2727	                    |          Internet
>   2728	                    |
>   2729	                 +-----+
>   2730	                 |     | Border Router
>   2731	                 |     |  (RPL Root)
>   2732	                 +-----+                      ^     |        |
>   2733	                    |                         | DAO | ACK    |
>   2734	              o    o   o    o                 |     |        |
> Strict
>   2735	          o o   o  o   o  o  o o   o          |     |        |
> Source
>   2736	         o  o o  o o    o   o   o  o  o       |     |        |
> Route
>   2737	         o   o    o  o     o  o    o  o  o    |     |        |
>   2738	        o  o   o  o   o         o   o o       |     v        v
>   2739	        o          o             o     o
>   2740	                          LLN
>   2741
>   2742
>   2743
>   2744	Thubert, et al.          Expires 28 January 2022
> [Page 49]
>   2745
> 
> 
>   2746	Internet-Draft               DAO Projection
> July 2021
>   2747
>   2748
>   2749	                Figure 12: RPL Non-Storing Mode of operation
>   2750
>   2751	   Based on the parent-children relationships expressed in the
> non-
>   2752	   storing DAO messages,the Root possesses topological
> information about
>   2753	   the whole network, though this information is limited to the
>   2754	   structure of the DODAG for which it is the destination.  A
> packet
>   2755	   that is generated within the domain will always reach the
> Root, which
>   2756	   can then apply a source routing information to reach the
> destination
>   2757	   if the destination is also in the DODAG.  Similarly, a
> packet coming
>   2758	   from the outside of the domain for a destination that is
> expected to
>   2759	   be in a RPL domain reaches the Root.
>   2760
>   2761	   It results that the Root, or then some associated
> centralized
>   2762	   computation engine such as a PCE, can determine the amount
> of packets
>   2763	   that reach a destination in the RPL domain, and thus the
> amount of
>   2764	   energy and bandwidth that is wasted for transmission,
> between itself
>   2765	   and the destination, as well as the risk of fragmentation,
> any
>   2766	   potential delays because of a paths longer than necessary
> (shorter
>   2767	   paths exist that would not traverse the Root).
>   2768
>   2769	   As a network gets deep, the size of the source routing
> header that
>   2770	   the Root must add to all the downward packets becomes an
> issue for
>   2771	   nodes that are many hops away.  In some use cases, a RPL
> network
>   2772	   forms long lines and a limited amount of well-Targeted
> routing state
>   2773	   would allow to make the source routing operation loose as
> opposed to
>   2774	   strict, and save packet size.
> 
> Maybe ad to be more explicit:
> 
> And example of such well-targeted routing state is what is needed to reach
> the
> neared loose steerig hops.
> 
>   2774	   Limiting the packet size is directly
>   2775	   beneficial to the energy budget, but, mostly, it reduces the
> chances
>   2776	   of frame loss and/or packet fragmentation, which is highly
>   2777	   detrimental to the LLN operation.  Because the capability to
> store a
>   2778	   routing state in every node is limited, the decision of
> which route
>   2779	   is installed where can only be optimized with a global
> knowledge of
>   2780	   the system, a knowledge that the Root or an associated PCE
> may
>   2781	   possess by means that are outside of the scope of this
> specification.
>   2782
>   2783	   This specification enables to store a Storing Mode state in
>   2784	   intermediate routers, which enables to limit the excursion
> of the
>   2785	   source route headers in deep networks.  Once a P-DAO
> exchange has
>   2786	   taken place for a given Target, if the Root operates in non
> Storing
>   2787	   Mode, then it may elide the sequence of routers that is
> installed in
>   2788	   the network from its source route headers to destination
> that are
>   2789	   reachable via that Target, and the source route headers
> effectively
>   2790	   become loose.
>   2791
>   2792
>   2793
>   2794
>   2795
>   2796
>   2797
>   2798
>   2799
>   2800	Thubert, et al.          Expires 28 January 2022
> [Page 50]
>   2801
> 
> 
>   2802	Internet-Draft               DAO Projection
> July 2021
>   2803
>   2804
>   2805	A.2.  Transversal Routes
>   2806
>   2807	   RPL is optimized for Point-to-Multipoint (P2MP) and
> Multipoint-to-
>   2808	   Point (MP2P), whereby routes are always installed along the
> RPL DODAG
>   2809	   respectively from and towards the DODAG Root.  Transversal
> Peer to
>   2810	   Peer (P2P) routes in a RPL network will generally suffer
> from some
>   2811	   elongated (stretched) path versus the best possible path,
> since
>   2812	   routing between 2 nodes always happens via a common parent,
> as
>   2813	   illustrated in Figure 13:
> 
> That figure needs to be improved if anyone who doesn't know what a
> transversal
> route is is expected to understand it. Just paint a picture with the
> minimum
> number of nodes to show a transversal route not part of the DODAG.
> Figure 14 is kinda ok., but in 13 its really unclear where traffic flows
> and why.
> 

I reworked a bit and called them East West to align to RAW

> Also: Is there any RPL messaging that would allow for the Root to have
> enough
> knowledge about transversal connectibity not part of the DODAG, which it
> could
> then tell the PCE ? If not, then for fairness, it would be good to
> highlight this as
> a gap for the solution.

We use siblings Info option

> 
>   2815	   *  In Storing Mode, unless the destination is a child of the
> source,
>   2816	      the packets will follow the default route up the DODAG as
> well.
>   2817	      If the destination is in the same DODAG, they will
> eventually
>   2818	      reach a common parent that has a route to the
> destination; at
>   2819	      worse, the common parent may also be the Root.  From that
> common
>   2820	      parent, the packet will follow a path down the DODAG that
> is
>   2821	      optimized for the Objective Function that was used to
> build the
>   2822	      DODAG.
>   2823
>   2824	   *  in Non-Storing Mode, all packets routed within the DODAG
> flow all
>   2825	      the way up to the Root of the DODAG.  If the destination
> is in the
>   2826	      same DODAG, the Root must encapsulate the packet to place
> an RH
>   2827	      that has the strict source route information down the
> DODAG to the
>   2828	      destination.  This will be the case even if the
> destination is
>   2829	      relatively close to the source and the Root is relatively
> far off.
>   2830
>   2831
>   2832	                      ------+---------
>   2833	                       |          Internet
>   2834	                       |
>   2835	                    +-----+
>   2836	                    |     | Border Router
>   2837	                    |     |  (RPL Root)
>   2838	                    +-----+
>   2839	                       X
>   2840	                 ^    v   o    o
>   2841	             ^ o   o  v   o  o  o o   o
>   2842	            ^  o o  o v    o   o   o  o  o
>   2843	            ^   o    o  v     o  o    o  o  o
>   2844	           S  o   o  o   D         o   o o
>   2845	           o          o             o     o
>   2846	                             LLN
>   2847
>   2848	       Figure 13: Routing Stretch between S and D via common
> parent X
>   2849
>   2850	   It results that it is often beneficial to enable transversal
> P2P
>   2851	   routes, either if the RPL route presents a stretch from
> shortest
>   2852	   path, or if the new route is engineered with a different
> objective,
>   2853
>   2854
>   2855
>   2856	Thubert, et al.          Expires 28 January 2022
> [Page 51]
>   2857
> 
> 
>   2858	Internet-Draft               DAO Projection
> July 2021
>   2859
>   2860
>   2861	   and that it is even more critical in Non-Storing Mode than
> it is in
>   2862	   Storing Mode, because the routing stretch is wider.  For
> that reason,
>   2863	   earlier work at the IETF introduced the "Reactive Discovery
> of
>   2864	   Point-to-Point Routes in Low Power and Lossy Networks"
> [RFC6997],
>   2865	   which specifies a distributed method for establishing
> optimized P2P
>   2866	   routes.  This draft proposes an alternate based on a
> centralized
>   2867	   route computation.
>   2868
>   2869	                 ------+---------
>   2870	                       |          Internet
>   2871	                       |
>   2872	                    +-----+
>   2873	                    |     | Border Router
>   2874	                    |     |  (RPL Root)
>   2875	                    +-----+
>   2876	                       |
>   2877	                 o    o   o    o
>   2878	             o o   o  o   o  o  o o   o
>   2879	            o  o o  o o    o   o   o  o  o
>   2880	            o   o    o  o     o  o    o  o  o
>   2881	           S>>A>>>B>>C>>>D         o   o o
>   2882	           o          o             o     o
>   2883	                             LLN
>   2884
>   2885	                   Figure 14: Projected Transversal Route
>   2886
>   2887	   This specification enables to store source-routed or Storing
> Mode
>   2888	   state in intermediate routers, which enables to limit the
> stretch of
>   2889	   a P2P route and maintain the characteristics within a given
> SLA.  An
>   2890	   example of service using this mechanism oculd be a control
> loop that
>   2891	   would be installed in a network that uses classical RPL for
>   2892	   asynchronous data collection.  In that case, the P2P path
> may be
>   2893	   installed in a different RPL Instance, with a different
> objective
>   2894	   function.
>   2895
>   2896	Authors' Addresses
>   2897
>   2898	   Pascal Thubert (editor)
>   2899	   Cisco Systems, Inc
>   2900	   Building D
>   2901	   45 Allee des Ormes - BP1200
>   2902	   06254 Mougins - Sophia Antipolis
>   2903	   France
>   2904
>   2905	   Phone: +33 497 23 26 34
>   2906	   Email: pthubert@cisco.com
>   2907
>   2908
>   2909
>   2910
>   2911
>   2912	Thubert, et al.          Expires 28 January 2022
> [Page 52]
>   2913
> 
> 
>   2914	Internet-Draft               DAO Projection
> July 2021
>   2915
>   2916
>   2917	   Rahul Arvind Jadhav
>   2918	   Huawei Tech
>   2919	   Kundalahalli Village, Whitefield,
>   2920	   Bangalore 560037
>   2921	   Karnataka
>   2922	   India
>   2923
>   2924	   Phone: +91-080-49160700
>   2925	   Email: rahul.ietf@gmail.com
>   2926
>   2927
>   2928	   Matthew Gillmore
>   2929	   Itron, Inc
>   2930	   Building D
>   2931	   2111 N Molter Road
>   2932	   Liberty Lake,  99019
>   2933	   United States
>   2934
>   2935	   Phone: +1.800.635.5461
>   2936	   Email: matthew.gillmore@itron.com
>   2937
>   2938
>   2939
>   2940
>   2941
>   2942
>   2943
>   2944


Whao, that was a huge one Toerless. You've been incredibly deep in your understanding and yes, you were correct all the way.

I made a great many changes an lotta reorg. The diffs will be hard to evaluate.

Again, many thanks!

Pascal