Re: [Coin] Last Call for the use case draft

["Kunze, Ike" <[e-mail]>]

Hi Dave,
 
Thank you again for your detailed review.
We believe that we have now addressed all of your comments as described below.
I have merged your two emails and we tag our comments with Use Case Draft Authors [UCDA].
Note that I have also added my earlier comments for completeness.
Please let us know should any of the changes not work for you.
You can find the latest version of the draft here: https://datatracker.ietf.org/doc/draft-irtf-coinrg-use-cases/05/
 
Thanks!
 
Cheers,
Ike

> On 27. Nov 2023, at 16:39, David R. Oran <[e-mail]> wrote:
> 
> On 8 Nov 2023, at 10:37, JEF wrote:
> 
> Hi group,
> 
> The use case draft is now in the status of "RG last call". Please review it and send your comments to this list. Our preliminary plan is to close this LC in 2 weeks after this IETF meeting: that is 25th Nov (anytime-on-earth 23:59). Then there will be a decision if we move it to the next stage: IRSG Review. 
> 
> Below is the link to this draft.
> 
> https://datatracker.ietf.org/doc/draft-irtf-coinrg-use-cases/
> Best regards,
> J/E/M
> 
> I am going through the use cases draft for last call and have some comments. I got through section 4.2 before running out of time. I hope to have time for the rest of the doc in the next few days, but I wanted to get these in now since the last call closing data has just passed.
> 
> As a general comment I don’t think the sections with requirements language add much of anything and could be eliminated without sacrificing the key points of the use case document. I also have a nebulous concern about how the use of requirements language in a research oriented informational RFC might be perceived by the IESG and others. I don’t feel strongly about this though - there may be benefits I’m not seeing to being explicit about normative requirements for use cases.
> 
[UCDA] We have toned done the requirements language by lowercasing the BCP 14 keywords and have repositioned the requirements as “desirable capabilities”. 
> Intro
> 
> what do you mean by “fields that require more than best-effort forwarding”? Surely not the usual meaning of “physics”, “economics” etc. Perhaps you mean “application domains” or something similar? You use the word “field” in a couple of other place too that you might want to rethink.
[UCDA] Changed to “application domains".
> in addition to improving performance, you might mention resilience to various failures as well as a motivation.
[UCDA] Added resilience as an additional motivation. 
> Section 3.1
> 
> beginning of 3.1.1 s/The scenario/This scenario/
[UCDA] Fixed.
> expand VR to “Virtual Reality” on first use of the acronym
[UCDA] Fixed.
> why is “display” in quotes?
[UCDA] Now used ‘COIN program’ consistently throughout the description, instead of the word ‘function’, which should make it more obvious that “display” is one such program.
> sensors can be in the hands or other body parts as well as in the headset
[UCDA] Added this to the subsentence.
> what is a “PND” (and please expand all acronyms on first use)
[UCDA] Added the expanded form as well as an explicit pointer to the terminology document.
> you seem to limit this use case to a single user (until a quick aside in the last paragraph), but there’s interesting current research on how to optimize 3D panoramic video across multiple users in the same virtual space, particularly by doing stuff in either access points (multiple uses in same home) or CDN edge nodes (for users in the same geography). It goes beyond simply doing multicast. Perhaps generalize a bit in the overall description?
[UCDA] Expanded the last sentence in 3.1.2 a bit more towards this as a possible extension. 
We think it is easier to describe the single user case first and then envision the “processing” function to be extended towards multi-user input with generating a single joint rendering towards more than one “display” program. 
> is the fact that the example in Figure 1 uses SDN relevant at all? Seems orthogonal to the important characteristics of this use case.
[UCDA] Denoted this in the text as an example for a programmable switch network
> in 3.1.3 you introduce “micro-service” without definition or references, and say that your use case somehow utilizes/requires them. Why is this important - could regular container/VM approaches work jut as well? Perhaps the point you want to make is for limiting state in the COIN intermediaries?
[UCDA] Changed this respective sentence, removing the implication that we envision to use microservices per se.
> in 3.1.4 the opportunities listed seem broader than COIN, and the service routing techniques are already used with all the services are hosted on “conventional” endpoints. It would be useful to try to get across where service routing changes or winds up being more important when there are COIN intermediaries compared to just a sea of endpoints.
[UCDA] Added some reference to involvement of programmable intermediaries in service routing and also added text to other opportunities to show relevance to COIN.  
For the first two, we agree that those opportunities already exist for deployments hosting merely at ‘conventional endpoints’ but we see them still apply in a distribution that involves also PNDs.
> in 3.1.5, RQ 3.1.7 has a simple answer. It’s infeasible. Constraint-base routing is sufficiently computation/state heavy that it has to be done either offline or in a high resource control plane server/service. Think you mean “how to instantiate forwarding state computed by constraint-based routing in the data plane and execute it at wire rate”.
[UCDA] Added two references, one applying programmable forwarding and the other one using multi-constraint routing, slightly refining the wording since, to us, the wording “constraint-based routing decisions’ already points to the establishment of suitable forwarding state as an outcome of, e.g., a routing protocol or a controller action towards the forwarder.
> 
> Section 3.2
> 
> in 3.2.2, what do you mean by “stream synchronization”. Is it minimal time dispersion in rendering?
[UCDA] Changed to "multi-stream aggregation”.
> saying “ideal to profit from some COIN” is a bit strong here as there might be alternatives as good or better. Perhaps say “attractive to profit from some COIN”.
 [UCDA] Changed to "XR stands to benefit significantly from computing capabilities in the network"
> there’s a bit of chicken/egg issue when you talk about “across different nodes on the path” since the selection of the path may be co-dependent on where the resources are. We’ve seen this in multiple contexts discussed in COIN and elsewhere about how to jointly optimize compute placement and routing. Perhaps bring this up explicitly?
[UCDA] Changed to "COIN functions can enable the distribution of the service components across different nodes in the network."
> 
> in 3.2.3 s/very little truly interactive/very few truly interactive/
[UCDA] Changed to "very few interactive"
> in 3.2.4 please be explicit on where you are talking about one way latency versus RTT. I assume your reference to 20ms is the latter?
[UCDA] Updated.
> in 3.2.5 RQ 3.2.3 I’m hardly convinced that the problems of CPU/GPU interoperability are in scope for COIN.
[UCDA] Removed the RQ
> in 3.2.5 RQ 3.2.4 What do you mean by “joint learning”. It’s not a term I’m familiar with. I do know about “split learning” and “federated learning”. Perhaps a reference if it’s just me that doesn’t know the term?
[UCDA] Changed to federated learning.
> 
> Section 3.3
> 
> in 3.3.1 you position this as “performers and audience are in different physical locations” Gee, we’ve had this since the early days of radio! :-) Much more interesting is “the performers are distributed across multiple physical locations”.
[UCDA] Even though the real-time audience feedback and viewer personalization aspects are new and challenging features beyond traditional broadcasting, 
we have extended the use case to include performers in multiple locations.
> 
> some of the aspects you draw out here are the same as for XR; perhaps explicitly cross-referencing would be helpful so the reader knows when you’re bringing out something new or unique to a use case as opposed to something common (or perhaps universal in which case looking at an individual use case doesn’t add anything).
[UCDA] Repositioned the use case as being a deeper dive into a concrete XR scenario to draw some specific requirements and challenges.
> in fact, I didn’t find this use case as separate from XR to be interesting or compelling at all. Would rather you tackled the problem of integrating a performance from performers in multiple locations.
[UCDA] Downplayed the features that are similar to traditional non-interactive broadcasting and included the challenges of personalization of productions from geographically-distributed performers.
> 
> Section 4.1
> 
> In 4.1.2, some challenging cases require sensor feedback in the microsecond range. Examples include controlling interferometers (e.g telescopes, LIGO-like systems). Would be useful to explicitly either include or exclude such things from this use case.
[UCDA] Named and explicitly excluded this group of use cases.
> I’m not sure whether the switch from PLCs to more software-focused control devices has any effect on the things you want to cover in this use case.
[UCDA] Changed the focus to the spatial decoupling of the process control and the controlled process.
> 
> Section 4.2
> 
> in 4.2.2 what is “packet-based sub-sampling”?
[UCDA] Added text to clarify this.
> what is “processed using manifold computations”? surely not from algebraic topology? Aren’t these just “computations”?
[UCDA] Changed  to “using various forms of computation”.
> I don’t get why stream processing systems are somehow limited to running on big sever machines. Often they can do multi-stage analysis and filtering, much of which could be outsourced to COIN system intermediaries.
[UCDA] Broadened the discussion of stream processing systems and added the opportunity for COIN to augment such systems. 
> In 4.2.5, many of those requirements are general and not specific to this use case (particularly those related distributing and coordinating COIN programs).
[UCDA] Added a corresponding disclaimer with an explicit reference to stream processing systems which are the main focus of that section.
> 
> Section 4.3
> 
> Not really a criticism on the introductory text, which is fine, but my strongest memory on factory floor safety was the bright red tape marking on the floor surrounding industrial robots, which indicated the maximum physical reach of the device. The person giving the tour pointed these out as the MOST important safety feature by saying that he could re-program the robots from his living room any time so by just watching the cameras and watching the victim, he could whack them with the robot if he wanted.
[UCDA] Added a sentence on “markings on the factory floor” and added the dimension of robots moving around.
> my take is that the software approach likely makes the environment LESS safe, not more. Perhaps one angle on using COIN approaches is that they can provide an independent IN PATH surveillance of software initiated actions to block unsafe commands.
[UCDA] Added sentences to mention possible new problems and also the independent surveillance aspect you mentioned. 
> Section 5.1
> 
> In the description, CDNs actually have two purposes. One is to reduce latency, but the other is to reduce load on origin servers. I suspect there are COIN angles to the latter as well as the former.
[UCDA] Added an explicit statement to that effect.
> A reference to switch-based load balancing (e.g. Silkroad https://dl.acm.org/doi/10.1145/3098822.3098824) would help justify this use case for the reader.
[UCDA] Added the reference as an addition to the FCDN reference.
> Under opportunities, I’m not sure why you introduce multicast (e.g. the [FCDN] reference) and then don’t follow it up with any elaboration on where COIN fits in with enabling or optimizing multicast approaches to CDN fanout.
[UCDA] Added a subsentence on that.
> I didn’t quite follow why you couch the applicability of service based routing in terms of routing to COIN program instances. It would seem the reverse is more compelling - using COIN instances to integrate service measurement data and then route from that COIN instance, to the selected service instance. What am I missing here?
[UCDA] Added the ‘from’ opportunity as additional opportunity to the service routing part.
> On a similar vein, the second opportunity talks about how something (you don’t say what) is doing the work to decide which COIN instance to use for something, as opposed to describing what the COIN instance would be doing to make the system better.
[UCDA] Rewritten to outline the choices as that of ‘service destinations’ where a COIN program instance may support the constraint-based selection of that choice of destination.
> On bullet three, I would have thought using multicast would reduce network utilization, not increase it.
[UCDA] Fixed.
> For the research questions, they seem to be couched in very general terms not specific to, or even oriented to, COIN issues. I’d suggest some re-formulation to target directly at COIN, such as how on-path intermediaries implemented in COIN instances can do the things mentioned in the research questions.
[UCDA] Added some more COIN-specific aspects to this.
> In RQ 5.1.3 joint optimization of compute and routing may or may not involve and constraint-based computations.
[UCDA] Added a wording that those constraints may support joint optimization (thus also leaving the aspect open that this can be achieved without).
> Section 5.2
> 
> In the description, why limit to Layer 2? in fact, I suspect nearly all the environments you mention use layer 3 topologies, not layer 2 (admittedly, early datacenter interconnects treated L3 as an overlay, but I’m pretty sure that’s no longer the case). For WAN connectivity, a similar evolution is happening.
[UCDA] Reworded to be more general in reference to the specific layer of the CFaaS
> The legacy requirement of compute interconnect necessitating direct layering over L2 protocols is rapidly becoming anachronistic. Certainly to my knowledge all the micro-service based approaches assume IP, not L2 connectivity. I’d suggest focusing this use case toward the future, to the past. This applies to the direct use of L2 multicast (as opposed to IP approaches like SSM) as well.
[UCDA] See previous comment
> You mention trust relationships, but then don’t follow that up with any further discussion. If the creation, maintenance, and enforcement of trust relationships is relevant to COIN it would be really useful to discuss that explicitly (my opinion - I think it is relevant).
[UCDA] Added some text in opportunities as well as a new RQ 5.2.4
> No comments on Section 5.3
> 
> Section 6.1
> 
> it would be helpful to separate inference from training, as the requirements are mostly distinct, and the effect on COIN approaches, especially in terms of memory load, are worth separate discussion.
[UCDA] Focussed the text on the training aspect only now. Added some text on the private data nature.
> in the training aspect, I suspect there is a good role for COIN approaches in optimizing back-propagation where there are a plethora edge clients performing inference.
[UCDA] Added an opportunity on this
> I’m not sure why you focus again on constraint-based routing in this use case. It may apply, but again I suspect that the actual constraint-based operations are not going to be performed by COIN programs - they are gong to execute the routing/forwarding/processing instructions that are the output of the constraint-based calculations. Perhaps just talk about COIN instances being the recipients of control-plane (or management plane) input to the state held by the COIN program to use in operating on its inputs.
[UCDA] Re-worded.
> What does RQ 6.1.2 have to do with this use case? I was scratching my head on why the AI applications in either training or inference generate “rapidly changing receiver sets”.
[UCDA] Added a reference to the dynamic selection of AI workers based on dynamically changing energy constraints.
> In RQ 6.1.4 what’s the relevance of HTTP to the AI use case?
[UCDA] Leftover from another case, revised now.
> Section 7 (security considerations)
> 
[UCDA] Restructured the entire section based on additional external feedback by Eric Wagner.
> At the risk of being laughed at maybe mention Functional or Homomorphic encryption as a workaround for cleartext data exposure to COIN intermediaries?
[UCDA] Added homomorphic encryption as one example in the new section.
> this might be a good place to talk about how applications may need to be cognizant of the presence of COIN intermediaries, and perhaps re-encode their data to do selective encryption that allows the COIN intermediaries to get at the pieces of data they need without exposing everything.
[UCDA] Added a paragraph talking about different forms of conveying information to the intermediaries along with corresponding risks.
> You need to talk about privacy as well. Not sure what exactly to say, but I’m sure you can come up with something reasonable.
[UCDA] Added a few notes on privacy.