Re: [icnrg] [irsg] IRSG review request draft-oran-icnrg-qosarch-04

"Schooler, Eve M" <eve.m.schooler@intel.com> Mon, 24 August 2020 16:04 UTC

Return-Path: <eve.m.schooler@intel.com>
X-Original-To: icnrg@ietfa.amsl.com
Delivered-To: icnrg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 285C63A108C; Mon, 24 Aug 2020 09:04:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.004
X-Spam-Level:
X-Spam-Status: No, score=0.004 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=intel.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l0WCojV0eRlE; Mon, 24 Aug 2020 09:04:07 -0700 (PDT)
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7CB183A0D72; Mon, 24 Aug 2020 09:03:57 -0700 (PDT)
IronPort-SDR: h7VQcBaCKtUncCYbVzgEavJwFWNwFgnLnRZTApN5kAp6rC3wf/TvVK+ZZr6OG9E/Jl/9ardPkh drJhXQgG8ZyQ==
X-IronPort-AV: E=McAfee;i="6000,8403,9722"; a="173971737"
X-IronPort-AV: E=Sophos;i="5.76,349,1592895600"; d="scan'208,217";a="173971737"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Aug 2020 09:03:56 -0700
IronPort-SDR: 4bQcBqdO4SzAUfS/5XQVjuI60HnFlHerCpjYqcSpGx95zeJZcs+/G3fGp3I/1SWwj1zm6+G5n5 f/wUuhCaxyYg==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.76,349,1592895600"; d="scan'208,217";a="328544015"
Received: from orsmsx603-2.jf.intel.com (HELO ORSMSX603.amr.corp.intel.com) ([10.22.229.83]) by orsmga008.jf.intel.com with ESMTP; 24 Aug 2020 09:03:56 -0700
Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 24 Aug 2020 09:03:55 -0700
Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Mon, 24 Aug 2020 09:03:55 -0700
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.173) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 24 Aug 2020 09:03:49 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EaZypoxsNaiDowaeJlXaXPT/w5juFeOjPtGuscWlQqUya3NumUUtkeUpWWIn0EMdhhhk1/xgYf6SnIZo6S5cEBfxliEQN+7n7LFOU3+4wofoF+pwRIeKwEz5WlpcLbWuE4qNy17tn8eg/3zWwKs9Tf+n7I+GHLGYJweRfyMZMWj84h0Xx5Nh06wNweq1wU7lQVEfZ3JDYQsM+W7o3tgxD4mT7+4uOJJWt6yEBsySzGF8zPX1uEEtbWO/hjKQ/Z0hQ5QOmjkxHU1+1Es+IRRGK00IR+TNDTLJR+npkbWAwJ3K+yPj3NoGCjnWYsqG5l2yWGadqb9VYfIVAsreXCRJhA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tcz31mdBWsCxsicQCStCULIP8/Tf0UJ6vBBYhJCQgO8=; b=E4F2MLMQFXReLDkL0hJD79apaJRy2h6HZumrDG6eFGrXn5FFRpRxB37px38/Zb5BM7EeYfYJ/93cxGmXI8ZnihP468I+q4KiT9h34X3HbMy65mlv9hDdi/l4tMSXVkPar8DYONr6ZvLHLadWG0f1Q+ZoXRwJraTN6lzEqhQghnolKAp9NAkNuyLnwNL9BBjPuN6zCkXJq9PL7QOu5EwvOM7v9INGDjAfOInFEmhzzeRjkkxQmD30QqSxpfsm75uqJ+EoZIi7hGPwRyaotwMZOIIBd4YTUaN0n0thn3zuOflKZSAbYh1fGBFGnn6enCKfWhz0OYoy/DC3p8mJlakw8w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tcz31mdBWsCxsicQCStCULIP8/Tf0UJ6vBBYhJCQgO8=; b=R5r3z2B1oWsd7RkYqrlSnXVX/PV304CXXvCBYIR3t1lMwqFDaQ4FD+StiqD/PNzXu4vZtW16HVjreS3UIZ/gy7CGGkWQJXyQB88Ktm+2aCGY+iHxKeTV9m9QMdNmGOlRH8HvII5SGhzguNI3vSkOIIAxKOSIXFErIRVp7ubI4oU=
Received: from SN6PR11MB3150.namprd11.prod.outlook.com (2603:10b6:805:d1::25) by SN6PR11MB2782.namprd11.prod.outlook.com (2603:10b6:805:63::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3305.26; Mon, 24 Aug 2020 16:03:48 +0000
Received: from SN6PR11MB3150.namprd11.prod.outlook.com ([fe80::9060:ff8d:64c3:d93d]) by SN6PR11MB3150.namprd11.prod.outlook.com ([fe80::9060:ff8d:64c3:d93d%7]) with mapi id 15.20.3305.026; Mon, 24 Aug 2020 16:03:48 +0000
From: "Schooler, Eve M" <eve.m.schooler@intel.com>
To: "daveoran@orandom.net" <daveoran@orandom.net>
CC: "icnrg@irtf.org" <icnrg@irtf.org>, "icnrg-chairs@ietf.org" <icnrg-chairs@ietf.org>, Colin Perkins <csp@csperkins.org>, Steering Group <irsg@irtf.org>, "Schooler, Eve M" <eve.m.schooler@intel.com>
Thread-Topic: [icnrg] [irsg] IRSG review request draft-oran-icnrg-qosarch-04
Thread-Index: AQHWSXJxJOQZcsiJTkyC2m9B69OKHakxSdfggABPTYCAAJvRcIAPcdEAgAYlzfA=
Date: Mon, 24 Aug 2020 16:03:48 +0000
Message-ID: <SN6PR11MB3150CE51DA1F7DF7D5096A1CD7560@SN6PR11MB3150.namprd11.prod.outlook.com>
References: <BB926E54-EFC9-42A6-A5B4-EA24A8AA9063@csperkins.org> <SN6PR11MB31500BBEBCE6A69A62F32A50D7440@SN6PR11MB3150.namprd11.prod.outlook.com> <E79F23E1-DBB1-4A96-977A-F563140F30D2@orandom.net> <SN6PR11MB31509302D11466B012F852D9D7450@SN6PR11MB3150.namprd11.prod.outlook.com> <10481907-3FDE-4922-854E-6954510DBA57@orandom.net>
In-Reply-To: <10481907-3FDE-4922-854E-6954510DBA57@orandom.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-version: 11.5.1.3
dlp-product: dlpe-windows
dlp-reaction: no-action
authentication-results: orandom.net; dkim=none (message not signed) header.d=none;orandom.net; dmarc=none action=none header.from=intel.com;
x-originating-ip: [99.4.120.204]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2ccf76dc-185c-4170-fdbb-08d8484749a8
x-ms-traffictypediagnostic: SN6PR11MB2782:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <SN6PR11MB27826C027C2817D2EA4FF3A9D7560@SN6PR11MB2782.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7691;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: VYtKD/HmR20SeAaYoVdvm6OQAMAFaNQwErEXoMPJLrG2N/e8i/tnSqt5hp+pV/X9r4FnpQvxT6Qy8j9efpKnr+qISDjbc/T/heBng3lRNmp25UvjAjXZYrMvQx6scBP1fMWAnxaBl8RjgUjxLa34FKyzzoFzZKm6cn8Jvj6mdoSEHVrNMbr2JpsFMlfpuM+wzFRdDRZ2el3QiKYnamGO7KjikRim4WGyvfL2th78EpvmQ+Ps3DZN41t6V6uv3O0+aPUIAMMLNH7zITQ/CCUIu+1B6LyMQUe5ev4aoIM6BsSM4DGu0dy15ZO5/hpgMBXLTt9SkXH1aCEx4MIBnWAF5pNgSLMZiIijda/Upx4fRZCy8IAQOGBvbN8UJyE2wWghdU6xg/x5IIBUbHCb33GNYQ==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3150.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(366004)(39860400002)(346002)(376002)(966005)(8936002)(64756008)(66446008)(66946007)(30864003)(9326002)(66476007)(66556008)(86362001)(6916009)(55016002)(7696005)(8676002)(54906003)(76116006)(71200400001)(4326008)(166002)(186003)(33656002)(53546011)(6506007)(5660300002)(2906002)(83380400001)(26005)(52536014)(478600001)(316002)(9686003)(107886003); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: irIKgA7joZRphJJXrxqlQ6fIdbck19RS6MAy+w1zyQcwACjsLMoUK7pECVT98orVGyaDgJvW49W24WM/rcBWic0SUL0ETnOxBUzLUo8OusRt9KGwthltXyvzSje0EgdkR1CmbTywuYzs1AH/fW/1+T9/MawpMEq2vji3fDEBczvpkWi14VSAmeCCN7r9+yjHtZXQtgtgrruO8eVE14bJKHIZflx/H0gYVNgVirjwDIOcKQeXyT3HmkXRnXSbg2bUcXl58jZb6Ebk/by23gLgoD0vJ40loALGgrf+vhl+Iq5ATDcKZ9V4rCikYc27XQrnjfGSdAsvbW5rFomxi1L2QFu0pa2rvQIrUoubTizyQz5/+aeCea5CY4xz32fDRov+l+yk7v+fXN2IGWJW0SSpCTTv9uHAkCWzxXk9lF0zbJR1XaGOfAGl9ax/rLgQf2gjQI0i3JSFNHL5rWLjKkvmYSx711CmUI1lAqXp4N1OUctHTeIWEGkcwu7zg7A20UVIZZM+Gs6gJmGy4kwA0PBvXHgKZp5j6V9YjJh4YzFxwhiBI/AVglHfSs9X1n6n+DenbcoqN8TS8bv+GIS5DarkVwbxc6nfxwLIr6Q6m0qudDhmyBQDTnCHQx+MMUO3yZmKhLoDuX8xUruIFMlfdleQuw==
Content-Type: multipart/alternative; boundary="_000_SN6PR11MB3150CE51DA1F7DF7D5096A1CD7560SN6PR11MB3150namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3150.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ccf76dc-185c-4170-fdbb-08d8484749a8
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Aug 2020 16:03:48.7645 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: VLJrGiinbZ9ouLFcFOVmGkiQxY0V6w6SfRI0NQ88VrIWUYHuzYStLQTF5jtg7VFR/5DzcIu4YIM1376YnibJT76AdB8Jny8w2aVb/xDFGPg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2782
X-OriginatorOrg: intel.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/icnrg/MsUTjIjs65QddWch4dsyOVlEJV8>
Subject: Re: [icnrg] [irsg] IRSG review request draft-oran-icnrg-qosarch-04
X-BeenThere: icnrg@irtf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Information-Centric Networking research group discussion list <icnrg.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/icnrg>, <mailto:icnrg-request@irtf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/icnrg/>
List-Post: <mailto:icnrg@irtf.org>
List-Help: <mailto:icnrg-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/icnrg>, <mailto:icnrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Aug 2020 16:04:18 -0000

Hi Dave,
Thanks for your articulate and thoughtful responses, especially your explanations of on- versus in-network computation, and of the QoS and layering principles. Your plan to address the more editorial comments (see previous e-mails) in your next version sounds like the right plan.
I look forward to your updated draft,
Eve


From: David R. Oran <daveoran@orandom.net>
Sent: Thursday, August 20, 2020 11:08 AM
To: Schooler, Eve M <eve.m.schooler@intel.com>
Cc: icnrg@irtf.org; icnrg-chairs@ietf.org; Colin Perkins <csp@csperkins.org>; Steering Group <irsg@irtf.org>
Subject: Re: [icnrg] [irsg] IRSG review request draft-oran-icnrg-qosarch-04


Thanks a bunch for these substantial thoughts on areas the document might address but currently doesn’t. This message definitely qualifies for TL;DR, but if it does generate a spirited discussion on the ICNRG list I think that would be a good thing (this message cc’s the IRSG since Eve’s original message with technical comments on the QoSArch document included the larger community, but I encourage further follow-up to stay on the ICNRG list).

I have a rather narrow view of what QoS belongs in the network layer (L3) as opposed to higher in the stack and I think that colors my thoughts on these issues pretty strongly.

In particular, I think there are two themes to this:

1) whether “in network computation” is appropriate to represent in the ICN inter-networking protocol (NDN or CCNx). One view is that its only real role is to deliver Interests to endpoints (or caches) holding the desired named content and returning the corresponding data, Alternatively one could consider resources for computational tasks beyond to this forwarding/routing function in scope.

2) whether the resources addressable using QoS treatments are just the “abstract” resources represented by protocol functions, or more directly the underlying resources used by the protocol to achieve those functions. For example, PIT capacity, forwarding packet rate, and cache capacity would be what were manipulated by requested QoS treatments, while things like CPU, DRAM memory, or energy consumption would not.

It would be really interesting to get people’s views on this. My own view, clearly represented by the current text of the QoSArch document is that for (1) it’s only the role of getting Interest and Data delivered that is relevant for ICN network-layer QoS, and for (2) only the abstract resources are what gets modeled.

Some more detailed thoughts are embedded.

On 10 Aug 2020, at 20:46, Schooler, Eve M wrote:

+ ICNRG (meant to cc originally, but I seem only to have cc’d the chairs)

Hi Dave,

I think the document is sound technically, at least in terms of what it discusses from the ICN perspective.

However, one aspect about QoS that seems less clear from the document (but that you hint at in various places, yet never seem to really sink your teeth into fully) is how does one specify QoS treatments, when the QoS spans multiple kinds of resources, not only the network but also storage/caches, compute resources, and energy (e.g., availability/excess clean energy):

Well, the document doesn’t reach the goal of being an actual specified QoS architecture. I’m just trying to point in a particular direction. So, the draft talks about the ability to specify richer QoS treatments than can be specified for IP, and to be able to represents tradeoffs as well as absolutes (e.g. sacrifice latency in order to get higher satisfaction rate). It doesn’t come anywhere close to saying exactly what treatments might be included or exactly how they are achieved algorithmically.

My goal is that the ICNRG and researchers in general will sink their teeth into this and that the QoSArch document, if published, can serve as pointing in a productive direction.

* section 4, Table 2: when you identify the kinds of resources ICN requires, there is mention of resources for Compute capacity for forwarding functions, but not for in-network computation.

I tried to clarify above that this is intentional. I think allocating compute resources via the L3 forwarding protocol would be a mistake and a pretty dramatic layer violation (although sometimes layer violations are called for and result in important advances, particularly if the violated layer boundary was poorly chosen by the architects). If you look at some of our current work like CFN (https://dl.acm.org/doi/10.1145/3357150.3357395) the resource management for placing and allocating compute is a protocol running on top of the ICN protocols, not inside as QoS-oriented extensions. What it could do for example (but doesn’t since current protocols don’t have the hooks) is use QoS machinery to ensure ongoing computations get higher reliability for result delivery than initiating new computations, or to rank less important computations to have less stringent latency bounds.

* section 5: this discusses TCP/IP equivalence classes and Intserv and Diffserv, but made me wonder if there is a similar discussion that could be had about QoS from the standpoint of computation engines, e.g., Kubernetes or other orchestrated workload/offload frameworks.

Given things like Kubernetes, Istio, and friends that place computations, there is certainly lots of work to be done to design joint optimization of routing, network capacity, and compute placement/load. The question for me is what inherent QoS treatments (and associated algorithms) in the L3 protocols are needed in order to enable higher layer resource management protocols to have the information they need, and the knobs to turn, in order to to that. My guess (although I can’t prove it) is that the needed capabilities in the L3 ICN protocols are quite modest, and are captured by the QoSArch document reasonably well. If that’s not the case then I’d really love to hear what might be missing or wrong in the overall approach I recommend.

One big open question that I do talk about in the document is whether Intserv-like traffic control is useful and if explicit admission control is needed. In the IP world, things worked out that supplying traffic control and admission control directly to consumer end hosts was a colossal no-op/failure, but that using those capabilities internal to individual AS’s to do traffic engineering turned out to be quite widely employed.

The provisional conclusion I put in the document is that it would not be too difficult to add traffic control to the protocols, but that adding admission control would be tricky at best and horrendously complicated at worst because of the multi-path/multi-destination forwarding model and the topological independence of application names.

* Section 6.3: Interesting tradeoff questions are raised between reliability vs latency vs bandwidth. Tradeoff algorithms would seem to need joint-optimization expertise, and so we will likely need the kind of work that folks like Edmund Yeh (Northeastern) and Andrea Goldsmith (Stanford) are doing on joint-optimization of caching and routing for heterogeneous wireless networks (and they’ve recently augmented their research focus to include joint optimization for caching-routing-and-compute).

Yes. I recommend that all those schemes operate at a higher layer than the L3 forwarding, but do influence the routing protocol in substantial ways. So, I would say that when we do work on ICN routing protocols this is the place to dig in rather than the data plane forwarding/caching primitives. Some prior work has tried to to the whole thing adaptively through stochastic forwarding algorithms (e.g. https://arxiv.org/abs/1505.05259). I’d need serious convincing that this direction has a lot of promise (nor its partner in crime, machine learning running in the data plane of switches to adapt forwarding and caching). I could easily be wrong here of course…

* Section 6.5: great discussion here of the interplay between network QoS and caching, from both consumers and producers.

Thanks!

* Section 7 and 7.1: the principles list does include caching as something needing further specification, but I wonder if 7.1 could do more to comment on the impact of computational resources for in-network compute (vs forwarding functions).

The dives straight into the core of what it means to have “in-network” computing as opposed to “on-network” computing. My own view is that if we don’t sort this out with reasonable architectural choices we’ll wind up recapitulating the less-than-stellar history of service chaining, VNFs, and the like.

In the specific context of ICN, for me some good guideposts are:

  *   If you need to transform data, you don’t do it in forwarders since the data is hashed and immutable
  *   If you need to look at payloads, you don’t do it in forwarders since the data is encrypted and you don’t have the keys
  *   If you want to compute in the same hardware box as the box that’s doing forwarding, that’s fine, but you are “on-network” rather than “in-network” since you are just a consumer/producer co-located with the forwarder.

Now, I actually think this is a good division, since most of the things people are trying to embed in the packet forwarding path of switches is really hard to do if it involves anything other than affecting the forwarding (e.g. firewall, DDoS mitigations, etc.). Now it my be that certain important optimizations of the computing functions running “on-network” can be helped by pushing expensive operations down to the switch hardware, just as people have been doing for years with smart NICs. That however, is not what I think people are proposing when they talk about “in-network” computing.

My thoughts on this topic are not fully formed, but I thank you for getting me to wonder aloud if this is the right document in which to raise these issues or if a broader QoS technical discussion and proposed guidelines should appear somewhere else. The reason why ICN has been an interesting place for this discussion is because of the close partnership with caching; my intuition is that if we can get QoS to comprehend not only network QoS but caching QoS, then it might help us to specify QoS and QoS tradeoffs among other a range of resources.

Best regards,
Eve

I hope this can generate some good discussion!

After going through these comments and thoughts, I don’t plan on making further changes to the current draft, so I’m going to submit it after waiting a day or two more for any further feedback on my proposed changes from the editorial comments.

Thanks again,

DaveO.