[PANRG] Review (was: Re: The PANRG RG has placed draft-enghardt-panrg-path-properties in state "Candidate RG Document")

[Theresa Enghardt <[e-mail]>]

Hi Med,

thank you for the detailed read and the extensive review! Your comments
are greatly appreciated.

I'm first replying to your general comments inline and then inserting
the specific comments and my responses at the bottom of this e-mail.


On 06.08.19 14:37, [e-mail] wrote:
> […]
>
> Some general comments: 
>
> * Having a terminology I-D may help as a support to architectural discussions, for example. Nevertheless, there is a risk of biased definitions in the absence of a reference architectural framework.

I think coming up with a unified architectural framework is a challenge,
as the research in PANRG touches upon different areas within the IETF,
including the Routing Area, Internet Area, and Transport Area, and
possibly more areas within and outside of the IETF.

Given that different areas may have different definitions of "Path"
(even within the same area, e.g., BGP paths vs. MPLS paths in Routing),
coming up with a unified terminology is a challenge.
In our internal Github wiki, we are collecting links to different
definitions and other references:
https://github.com/theri/draft-enghardt-panrg-path-properties/wiki -
Maybe this plus some discussion of the differences between their and our
definitions could become an appendix to our document at some point?
Feel free to point out more documents or work that you feel should be
included here.


> * The document implies some architectural assumptions that are not justified (e.g., domain vs. backbone, hosts assumed to be within the same domain as the access network, etc.) 

We introduced the classification into domain properties, backbone
properties, and dynamic properties in the hope that such classification
might make it easier to talk about different properties, use cases, etc.
However, right now I'm not sure if the current classification into
"domain" and "backbone" properties is indeed the right way to go

In the draft we give a number of reasons why it may matter if a path
element that has a specific property is in the same administrative
domain as a host or not. I'm sure there are considerations here that
already touch upon other research questions in the Questions draft,
e.g., the second question of Discoverability, Distribution, and
Trustworthiness of path property. However, the second question already
asks how an endpoint can get access to trustworthy path properties,
which we do not attempt to answer in the path properties draft.

I think here we should try to find the sweet spot between a narrow focus
on "just terminology" vs. an extensive discussion of where path elements
may be located, who may control them, and how this may influence path
properties and their availability, etc. I'm sure we can have useful
discussions here, which may have an impact on our definitions of path
properties, but we should also make a conscious decision of how much of
this to put in the path properties draft.

I'm very interested in getting input from the Research Group regarding
this question.


> * Some terms are already defined in existing RFCs, no need to define new ones (e.g., one-way delay, one-way delay variation, one-way loss, etc.). It is OK to list them, but with pointers to the authoritative RFCs. 

Yes, our intention is to list path properties, not to redefine a
property if it already exists (or something very similar exists). We
will add references to existing definitions.


> * The definition of some terms is not aligned with existing practices/techniques (ERO, PCE, SFC, for example). Take the example of "Path" which assumes "alternating between nodes and links". This assumption is not accurate for an SFC/SFP or a path presented as an ordered list including AS Numbers.

Regarding the ordered list of AS numbers:
Yes, we may want to align our definition with this or add more terms.
Right now we can express an entire network or AS as subpaths, e.g.,
consisting of all nodes and links within this AS. Maybe, we want an
additional "aggregated" path element (group of nodes and links? cloud?),
which is not a subpath, but an amorphous component (node?) on the path,
which may include multiple nodes and links. As we may not have full
visibility of all path elements in practice, from the point of view of a
host, most paths probably include at least one such "aggregated" path
element anyway.

Regarding SFC/SFP:
The definition for a path as path elements "alternating between nodes
and links" exists to prevent nonsensical paths, e.g., consisting only of
nodes or only of links. As a path may include multiple nodes on a single
physical host, we allow nodes and links to be virtual, e.g., to just be
a virtual switch or a virtual interface that packets can pass through.
Just to make sure I got your comment right: Here you are saying that
defining individual Service Functions (SF) or a Service Function Chain
(SFC) to be nodes (which may be virtual) between which links may exist
(either physical or virtual, allowing virtual links to not just be
between virtual network interfaces but even within a single host) does
not solve the problem, right? So the Service Function Path (SFP) cannot
just be a sequence of nodes and links? Do you think this is this
necessarily just a sequence of nodes, no links, because we would have to
stretch the definition of a virtual link too far, or is there a
different reason why alternating nodes and links does not work?


> My review of the draft is available at: 
>
> * pdf: https://github.com/boucadair/IETF-Drafts-Reviews/blob/master/draft-enghardt-panrg-path-properties-02-rev%20Med.pdf 
> * doc: https://github.com/boucadair/IETF-Drafts-Reviews/raw/master/draft-enghardt-panrg-path-properties-02-rev%20Med.doc 

Responses to the individual comments follow:

On "This document defines and categorizes information about Internet
paths that an entity, such as a host, might have or want to have.":

[Med1]: This assumes a global connectivity. I don’t think this
restriction is justified (consider enterprise networks, mesh networks,
etc.).
[TE1]: I agree - perhaps change "Internet paths" to "paths across a
network"?

[Med2]: Instead of reasoning about paths, what actually matters is the
connectivity service(s) provided via an upstream network.
[TE2]: Well, this is the Path-Aware Networking Research Group, so it
makes sense to reason about paths, right? Yes, as an endpoint you
actually care about what services your path provides, but I'm not sure I
want to change any text based on this comment.
Maybe add "[information … that an entity … might have or want to have]
about a path and the services it provides"?

On "This information is expressed as properties of paths between two hosts."

[Med3]: A host will have a partial visibility of the available path(s)
(and their associated services). The full visibility cannot be
established beforehand. Furthermore, such information is volatile (e.g.,
handover or multi-attachment with session continuity).
[TE3]: I agree, but I'm not sure I want to add text to the abstract
based on this. I don't think we imply full visibility here if we talk
about "information" that a host can get about paths. But we have to make
sure this point is clear in the body of the text.

[Med4]: GENERAL: the properties are dependent of the direction. Some
properties will apply for inbound, outbound, or both.
[TE4]: I agree, and I think our (planned) definition accounts for this:
Links are unidirectional, the reverse direction is another
unidirectional link between the same nodes, and a property applies on a
sequence of path elements.

[Med5]: [Between two hosts] excludes point-to-multipoint communications.
Consider the case of an STB which want to retrieve TV channels using
multicast.
[TE5]: Right now, we define path as between two hosts (will perhaps
change to "between two nodes" to include, e.g., overlays, but of course
the two nodes can be hosts), and then point-to-multipoint can be
expressed as a set of multiple paths, one for each pair of sender and
receiver. Then, the same property can apply to multiple paths and, thus,
be essentially the point-to-multipoint communication.
About the text of the abstract, perhaps changing this to "[properties of
paths] between two or more nodes" helps?

On "Because the current Internet provides an IP-based best-effort bit
pipe, hosts have little information about paths to other hosts.":
[Med6]: What is the causality effect here?
[TE6]: I think here we intend to say that, in the current Internet, the
path and its services are transparent to hosts, so hosts only know
something is reachable, but they do not get all the details of via which
networks, they do not necessarily get details on the services (e.g., QoS).
Do you have a proposal of what text to change here?

On "A Path Aware Network exposes information about one or multiple paths
through the network":
[Med7]: I do still believe this is a weird term: networking is about
manipulating paths, by essence!
[TE7]: Well, hosts do not usually directly manipulate paths, unless they
do source routing, right? Anyway, the term "Path Aware Network" is taken
from the questions draft, which is an adopted Research Group document,
so I wouldn't change it and introduce even more terms in the process.

[Med8]: Difficult to parse
[TE8]: Will see how we can rephrase this.

Crossed out second part of this sentence: "[It is impossible to provide
an exhaustive list of path properties], as with every new technology and
protocol, novel properties might become relevant"
[TE]: Is the comment here that the crossed out part does this not work
as an explanation of the first part of the sentence? Why not?

On "[Traffic policies: … Such policies can allow or disallow sending
traffic over specific networks or nodes":
[Med9]: This is about path selection covered in the 3rd bullet.
[TE9]: True, this can be a case of path selection - or you may find out
that your policy does not allow you to send your traffic at all. The
latter case is not covered by our current definition of path selection,
so I think mentioning it here makes sense. But we should rephrase this
sentence to make it more clear. Any suggestion?

On "[select a protocol depending on the capabilities of the] on-path
devices":
[Med10]: I agree with this example, but I have the following comments:
The selection of the protocol is not only specific to on-path devices,
but it can be the other way around: It is the selection of the protocol
which may infer the devices that will be involved in a path. Consider
the example of the 0-RTT Convert Protocol which is involved in a path
only when invoked by a host; such invocation will lead to the use of
MPTCP or TCPinc while this Is not doable with the default path. Another
example of traffic policies is a connection which may be composed of
multiple streams, each stream with specific requirements. A host may
decide to invoke a given service function (e.g., transcoding) only for
some streams while others are not processed by that service function.
[TE10]: True, there can be more complex interactions. Do you have a
suggestion of what text to add or change here?

On "the monetary cost [of the link]":
[Med11]: This is linked to a network, not a path.
[TE11]: Yes, this usually applies to only a part of the path, e.g., a
specific network. However, from the point of view of a host, which sees
the path containing this network, it may be a property of the path.
Perhaps we should change the sentence here to "the monetary cost to send
data across a network on the path" to make the example clearer.

On "Another example is an enterprise network where all traffic has to go
through a firewall, in which case the host needs to be aware of on-path
firewalls."
[Med12]: How this information is useful for enforcing “traffic policies”
discussed in this bullet? What would be interesting is to have means to
control such firewalls.
[TE12]: The idea here was that a host may have a policy to have all its
traffic filtered through a specific firewall, so if the firewall is not
available for some reason, it will not sends its traffic, or it will
send different traffic, e.g., encapsulate it using a VPN. Does this
example make sense to you?
Agree that controlling firewalls from the host could be another
interesting use case, but I'm not sure if this is in scope of this
draft. After all, our model is that the host gets information about the
path, not that it manipulates the path.

On "Network monitoring: Network operators can use path properties (e.g.,
measured by on-path devices), to observe Quality of Service (QoS)
characteristics of recent end-user traffic, and identify potential
problems with their network early on, before the end-user complains."
[Med13]: Which information is exposed here?
[TE13]: The idea here is to expose performance characteristics, such as
latency or packet loss, to nodes on the same subpath, so the performance
problem can be mitigated. Think LOOPS overlay nodes which might detect
an increase in packet loss and, as a result, retransmit packets or add
FEC. Perhaps "monitoring" is not the right term here, because this
scenario also involves mitigation - maybe "Performance enhancement" is a
better term here?

On "[a non-dynamic path property might stay relevant, e.g. a NAT can be
assumed to stay on a path during the lifetime of a connection, as the
removal of] the NAT would break the connection."
[Med14]: Not for MPTCP/QUIC.
[TE14]: Right, this was written with single-path TCP in mind.
Perhaps change this to "a non-dynamic path property might stay relevant
for a longer period of time, e.g. a NAT typically stays on a specific
path during the lifetime of a connection involving packets sent over
this path" - this leaves it open whether NAT rebinding occurs and
whether additional packets of the same connection are sent over
different paths. I think the only case in which the "typically" does not
apply here is if the NAT were to be removed from the specific path,
e.g., the NAT functionality of the router switched off. And this I find
unlikely to occur during most connections.

On "Non-dynamic properties are further separated into (local) domain
properties related to the first few hops of the connection, and backbone
properties related to the remaining hops. Domain properties expose a
high amount of information to hosts and strongly influence the
connection behavior while there is little influence and information
about backbone properties."
[Med15]: Advanced service (e.g., NAT, IDS, ..) may be located in a
Centralized PoP or hosted by a third party (DDoS protection, for
example). Do you consider these as domain properties?
[TE15]: In our current definition, they would not be domain properties.

[Med16], on "Domain properties expose a high amount of information to
hosts": Why?
[TE16]: The assumption here is that from nearby path elements, you may
be more likely to get performance information, such as Channel
utilization and such from QBSS information elements sent by a WiFi
access point, or that you are more likely to know about and explicitly
configure an HTTP proxy within your own administrative domain.

[Med17], on "strongly influence the connection behavior": ?
[TE17]: The assumption here is that, for example, performance may be
more strongly influenced by the access network because it includes the
bottleneck or a proxy. This doesn't always hold though, and perhaps we
need to revisit our assumptions.

[Med18]: You may need to call out your topology assumptions because some
conclusions may not apply to some topologies.
[TE18]: Agreed, and I think this leads us to a more general discussion
about the scope of the draft.

On "Host":
[Med19]: I personally prefer the definition in RFC1122 ["Host: ultimate
consumer of communication services. A host generally executes
application programs on behalf of user(s), employing network and/or
Internet communication services in support of this function."]
[TE19]: Thanks for the suggestion, I really like this definition, too.
The distinction based on "processes packets that are addressed/not
addressed to [a node]" has its problems, as this does not say on which
layer we consider the addresses, this gets more complicated with
encapsulation, etc. So I'm in favor of changing our definition of a host
to something along the lines of RFC 1122.

[Med20]: This is about receiving. You may also cover sending
[TE20]: We understand "processes packets" to include both sending and
receiving, but I suggest we switch to a different definition as
discussed above.

On "Path element: Either a node or a link."
[Med21]: This definition does cover, for example, the case of a path
identified by an ordered list
of AS Number. Check for example the definition of ERO in RFC3209, or
even in recent documents
such as RFC7570
[TE21]: True. ERO can include groups of nodes, possibly this can mean
entire ASes. As said above, we may want to allow "aggregated" path
elements that are entire networks, represented by the network's AS number.

On "Path: […] alternating between nodes and links":
[Med22]: This definition excludes recent notion such as SFP defined in
RFC7665
[TE22]: (Repeating the reply from above) Just to make sure I got your
comment right: Here you are saying that defining individual Service
Functions (SF) or a Service Function Chain (SFC) to be nodes (which may
be virtual) between which links may exist (either physical or virtual,
allowing virtual links to not just be between virtual network interfaces
but even within a single host) does not solve the problem, right? So the
Service Function Path (SFP) cannot just be a sequence of nodes and
links? Do you think this is this necessarily just a sequence of nodes,
no links, because we would have to stretch the definition of a virtual
link too far, or is there a different reason why alternating nodes and
links does not work?

On "A path can be viewed as an abstraction on a specific layer, omitting
lower layer path elements."
[Med23]: Contradicts with the sentence about alternating nodes/links.
[TE23]: Why? If we define a path on, e.g., layer 3, we can treat the
link between two layer 3 devices as a virtual link, so we abstract from
all physical links and nodes on layer 2. Do you have an alternative
proposal how to phrase this?

On "Property: A trait of one or a sequence of path elements […]"
[Med24]: I’d like to double check If the defining covers also the
concept of SFC/SFP. The path will be built to accommodate a property
(result of SFC).
[TE24]: I think this depends on how we define the sequence of path
elements, see above, to make sure we cover these concepts.

On "Aggregated property: A collection of multiple values of a property
into a single value, according to a function."
[Med25]: Note this cannot be applied to all properties of a connectivity
service.
[TE25]: Correct. For all properties of a connectivity service, you would
need a set of all aggregated properties.

On "Measured property: A property that is observed for a specific path
element or path, e.g., using measurements."
[Med26]: I prefer « Observed » rather that measured because properties
may not only be about traffic performance metrics.
[TE26]: True, we define lots of other properties for which we can
observe a value. So we may change this.

On "Estimated property: An approximate calculation [CROSSED: or
judgment] of the value of a property."
[Med27]: As above, this is too measurement-centric. This may not be
applicable to all properties.
[TE27]: The intention of the "or judgment" is to express that this may
not be a numeric and/or performance-related property. I understand
"estimate" to be broader than numerical but rather to include "an
educated guess". However, it seems to me like most words in this space
imply numeric values. So I'm looking for the right word here - does "An
approximate calculation or assessment of a numeric or non-numeric value
of a property" work? Any other suggestions?

On "3. Domain properties":
[Med28]: I’m afraid this section (and section 4) implies architectural
considerations which may not be appropriate in a terminology document.
[TE28]: As the scope of this document is to answer the first question of
the Questions draft (Section 2.1 in draft-irtf-panrg-questions), "how
are path properties defined and represented?", yes, this is a
terminology document, but I wonder if possible classification of path
properties is in scope. Here, our intention was to introduce such
classification (e.g., dynamic and non-dynamic) to make it easier to
reason about different properties, their usefulness, etc. Repeating what
I wrote above, I'm not sure if the classification into "domain" and
"backbone" properties is indeed the right way to go, and I'm very
interested to get input from the Research Group here.

On "the first few hops":
[Med29]: 2, 10 ?
[TE29]: We kept this intentionally open to cover both cases in we really
consider only the first hop and cases in which we consider the entire
access network behind that hop as well. I think deciding whether we want
a clear definition here depends resolving the issue raised by the
comment [Med28] above.

On "the same administrative domain as a host considering them."
[Med30]: L2, L3, ?
[TE30]: Personally I mostly think about domains in this document as
everything covered by the same AS, but yes, it would be very interesting
to come up with a clear definition of domain here.
For example, PCE defines a domain as a "collection of network elements
within a common sphere of address management or path computation
responsibility, can be IGP areas, ASes, multiple ASes within a Service
provider network. may also exist as sub-domains of areas or ASes.".
PANRG is more broad, but maybe we can come up with something similar here?
Again, I think deciding whether we want a clear definition here depends
resolving the issue raised by the comment [Med28] above.

[Med31]: This would exclude then a host connected to a UE connected to a
PLMN or a CPE connected to an enterprise or provider networks. These
elements do not belong to the same
administrative entity.
[TE31]: In my understanding, an element belonging to an administrative
domain does not necessarily mean that the element belongs to the entity
which owns the domain. For example, if my phone (the UE) connects to a
cellular network (the PLMN), the mobile phone still belongs to me.
However, the phone might also be a host on a path through this cellular
network. In this case, I intend the definition to have the phone, the
base station, and the cellular infrastructure behind the base station
all belong to the same administrative domain, as for example all L3
devices here get assigned IP addresses from a common address space that
belongs to the provider of the cellular network. If my phone leaves the
cellular network, it does not keep the IP address, then it's no longer
part of that administrative domain. Can we come up with a more general
definition of a domain that does not rely only on IP address space?
Again, I think deciding whether we want a clear definition here depends
resolving the issue raised by the comment [Med28] above.

On "Due to the potential physical proximity and pre-existing trust or
contractual relationships between hosts and path elements within the
same domain, domain properties may be more accessible to the host than
other properties."
[Med32]: Which contractual relationships would apply to this case?
[TE32]: For example, if my host connects to a service provider's
network, I may be a customer of this service provider, e.g., through my
DSL subscription. This MIGHT mean that the provider is more willing to
or has to give me (or my host) more information, e.g., about network
performance characteristics.
Physical proximity is easier, I think: A WiFi AP may send out
information of its currently observed channel utilization within QBSS
information elements. A host only gets this information because it's
directly connected to the AP, which implies physical proximity.
Again, such considerations already touch upon the second question of the
Questions draft.

[Med33], on the same text: I’m not sure about this.
[TE33]: This may not always be the case, naturally. We just thought it
might be helpful to state that it might be the case to motivate our
classification into domain and backbone properties.
Again, this depends on [Med28] resolution.

On "hosts may be able to influence […] which domain they are in"
[Med34]: Connecting to a domain does not mean that the host belongs to
that domain.
[TE34]: Then this is a wording issue and we should change something in
our definition to "connect to a domain" or "be in a domain" instead of
"belong to a domain".
However, again, this depends on [Med28] resolution.

On "they may be able to influence the properties of path elements within
this domain."
[Med35]: An example ?
[TE35]: Given in the rest of the paragraph: "when connected to an Access
Point, the user may move
closer to enable their device to use a different access technology,
potentially increasing the data rate available to the device."

[Med36]: How a host can influence the one-way delay provided by a network?
[TE36]: Probably not. We never suggest that a host can influence ALL
properties.

On "Access Technology: The physical or link layer technology used for
transmitting or receiving a flow on one or multiple path elements in the
same domain. The Access Technology may be given in an abstract way,
e.g., as a Wi-Fi, Wired Ethernet, or Cellular link. It may also be given
as a specific technology, e.g., as a 2G, 3G, 4G, or 5G cellular link, or
an 802.11a, b, g, n, or ac WiFi link. Other path elements relevant to
the access technology may include on-path devices, such as elements of a
cellular backbone network. Note that there is no common registry of
possible values for this property."
[Med37]: This is basically in a domain != from the local network
[TE37]: Why not in the local network? I think this specifically applies
to many local networks, where it's actually interesting if, e.g., a WiFi
AP supports 802.11ac or not. Perhaps I'm misunderstanding your comment.
Could you clarify, please?

On "[Monetary Cost: The price to be paid to transmit a specific flow
across a] subpath."
[Med38]: A network
[TE38]: Right now, in the context of a path, all path elements within
the network form a subpath. So this applies to the subpath. However, if
we extend our terminology to include aggregated/abstract path elements,
maybe we can have an abstract path element that represents a network.

On "Typically, backbone properties are less accessible to a host than
domain properties, due to the potential increased distance and the lack
of pre-existing trust or contractual relationship."
[Med39]: Not sure about this.
[TE39]: Not sure if "typically" is the right word here either, but,
similar to the part with [Med33], the intention is to say that this
might be the case.

On "hosts are less likely to be able to influence which path elements
form their path in the backbone, as well as their properties."
[Med40]: This may contradict with slicing use cases.
[TE40]: Sounds interesting. Could you point me to a specific reference,
please?

On "Presence of a certain [CROSSED: network] service function on the path":
[Med41]: To be aligned with RFC7665
[TE41]: Agreed.

On "Administrative Entity: The administrative entity, e.g., the AS, to
which a path element or subpath belongs."
[Med42]: This is an administrative domain, not * entity *
[TE42]: Isn't an AS also an entity? But yes, maybe we could have the
administrative domain be a property instead?

On "Disjointness":
[Med43]: you may also refer to the PCE RFCs which defines also the
notion of near disjointness
[TE43]: I found RFC 4655 saying "The path computation request may
include a significant set of requirements, including the following: […]
the number of disjoint paths required and whether near-disjoint paths
are acceptable", but it does not define what near-disjoint is. Is there
a specific degree of disjointness that is generally still considered
"near"? Anyways, I think if a host or other entity knows about the
degree of disjointness, it can decide whether this is "disjoint" enough.
Maybe we want to add a reference to PCE as an example use case.

On "Latency", "Latency variation", "Packet loss":
[Med44]: Please use conventional terms : one-way delay, one-way delay
variation, one-way loss, etc.
[TE44]: Yes, it might be useful to use terms as defined, e.g., in IPPM,
and add references here.


Again, thank you for the extensive feedback.

Best,
Theresa