[ippm] IPPM IOAM Side Meeting Minutes

[Tal Mizrahi <[e-mail]>]

====================================
IOAM Side Meetings
IETF 106, Singapore
Monday, November 18th, 2019, 08:30
Tuesday, November 19th, 2019, 12:00
====================================

Minutes: Tal Mizrahi.

This is a summary of two side meetings that took place in IETF 106.
The "Summary and action items" summarizes both meetings, and detailed
meeting minutes follow.
Please let me know if I missed anything.


Summary and action items:
=========================
- The IOAM design team will continue its bi-weekly calls. The next call
will be on December 4th. To be announced again a few days before the
meeting.

- Loopback flag:
  - The following clarifications will be added to the flag document:
    - Loopback is intended to be sent to the encapsulating node.
    - Loopback is only applicable in encapsulations where the encapsulating
(source) node is available in the header.
    - If an encapsulating node receives a looped back packet that was not
originated from the current node, the packet is dropped.
    - Loopback is intended to provide an accelerated alternative to
Traceroute.

- Active flag:
  - The following clarifications will be added to the flag document:
    - The active flag is intended to simplify the implementation of
decapsulating nodes, by indicating that the packet should not be forwarded
further.

- Data draft:
  - Some of the data fields have both a short format and a long format. It
will be raised to the working group whether this should be revisited.

- Data / DEX draft:
  - The current data draft and DEX draft are applicable to multicast. In
addition, it may be useful to have an additional draft that focuses on
multicast extensions.

- IPv6 options draft:
  - The text about dropping a packet with an IOAM option when the option is
not expected should remain as-is, and we are waiting to see if there are
further comments on the mailing list.

- IOAM profile draft:
  - The IOAM profile draft will cite the IOAM Yang model, as an IOAM
profile may be defined by a specific assignment to the Yang model. Also a
profile may augment the Yang model in order to define profile-specific
configuration.



============================
Monday, November 18th, 2019
08:30
Meeting Minutes
============================


Attendees
=========
Tal Mizrahi, Barak Gafni, Greg Mirsky, Carlos Pignataro, Al Morton, Roni
Even, Shwetha Bhandari, Rakesh Gandhi, David Melman, Justin Iurman, Fan
Yang, Richard Foote, Haoyu Song, Ramesh Sivakolundu, Xuesong Geng, Frank
Brockners.


Agenda bashing
==============
Tal: on our agenda we have: (1) open issues regarding the flag document,
(2) IOAM profile document.
Shwetha: we should also discuss open issues about the data document.
Tal: right. Starting with the flag draft.
Barak: I have sent a few comments about the flag draft. Most of the issues
are about the loopback flag. Will open issues on GitHub.
Tal: are there any other issues?


Loopback flag discussion
========================
Barak: loopback mode - as defined today it should collect data on the
return path. Complexity which is not needed. Not related to the path. This
is the major issue.
Frank: do you want it as an option, or excluded?
Barak: would prefer if it is excluded.
Shwetha: what needs to be changed in the flag?
Greg: the assumption is that there is a return path. In some encaps there
is no information about the sender node. The return path is not necessarily
possible. In active OAM there are special procedures to make sure you know
how to return a packet to the sender. For example, MPLS is an issue.
Barak: how do you respond to an ICMP?
Greg: it is an IP packet over MPLS. You can't assume in IOAM that the
packet is over IP.
Frank: we do not need to turn loopback into active OAM. We were assuming
that it is possible to loop back, but if that is not the case, then
loopback should not be used. It is applicable where the return path exists.
Greg: I cannot imagine a spec that says you can use it without saying where
it is applicable.
Frank: the current draft says it is applicable only if there is a return
path.
Barak: do we have a draft about IOAM over MPLS?
Rakesh: will be presented today.
Barak: that draft will need to define relevant functionality.
Roni: encapsulation drafts will need to define this.
Frank: each draft will have to say wheter loopback is applicable or not.
Greg: this is not the best approach. What happens in multi-guest networks?
Each transit node sends back to the sender? That is a lot of extra traffic
that you are generating in the network. You are choking the service.
Shwetha: this is not always on by default.
Greg: you are giving an operator a loaded gun.
Roni: it is just specifically a problem for multicast, where you will
define how such a service will work anyway.
Greg: first, it is a problem to collect on the return path. In multicast it
is also a problem, because transit nodes will do nothing.
Carlos: unicast is one way forward versus multicast which is multiple ways
forward.
Greg: it may be abused.
Shwetha: it can also be a problem with existing traceroute, if you flood
the network with it.
Greg: it is an inherent problem with IOAM.
Barak: Greg, can you suggest text for the security considerations?
Greg: will consider it.
Barak: the encapsualting node sets the loopback flag, but it is not clear
who is going to use it. If a top-of-rack applies the loopback, the host
will get the responses. We need to align who the edge is, and who is the
destination of the data.
Shwetha: today we just swap the source and destination.
Frank: we want to try not to over-engineer it.
Barak: we need the "source" to be the encap node.
Shwetha: depends on the encap.
Greg: this discussion is helpful. The scope of loopback assumes that the
encap is always IP?
Barak: no. For example it can be applicable to MAC addresses as well, with
the GRE encapsulation.
Shwetha: wherever we have a source and destination, that would work.
Greg: that is a problem. We should say it is applicable to encaps that
include a source and destination.
Shwetha: makes sense.
Barak: sure.
Tal: to summarize, the loopback is applicable in cases where we have a
source and destination?
Greg: applicable to where we have a source.
Barak: specifically when the loopback receiver is the source node.
Greg: not necessarily.
Barak: how do you manage it?
Greg: loss and delay measurement in MPLS specifies the address you have to
return to.
Barak: but MPLS is not in scope.
Greg: that was just an example. There may be a case where the returning
packet does not return to the original source.
Barak: the whole point of loopback is to get back to the encapsulating
node. To enhance the performance of traceroute.
Greg: if we care about complex networks.
Barak: that can be done in other ways.
Greg: what is the value of loopback?
Barak: if you need traceroute - this is a way to accelerate it.
Greg: what if it is an SDN set by an orchestrator?
Barak: orchestrator is not in our context.
Greg: loopback gets back to the sender.
Barak: tie the source with encap node.
Ramesh: only the source (encap) can initiate a loopback.
Barak: what happens when a source gets a loopback if he is not the
originator.
Greg: is there a way to know whether you generated it or not?
Barak: based on the source address.
Carlos: it should be dropped if you receive a loopback that is not from you.
Frank: right.
Barak: yes.
Tal: any other comments, Barak?
Barak: no.
Frank: what was the number of this issue on Github?
Barak: #138.
Tal: any other comments regarding the loopback flag?
No further comments.

Active flag discussion
======================
Greg: what is the use case of active flag? Who generates the message, who
uses it?
Barak: can you explain the question?
Greg: we already have existing active OAM protocols. Does it mean that in
these use cases the active flag is set?
Barak: two different use cases.
Greg: I am asking. What is the use case? Is the active flag set by the
encap node?
Barak: yes.
Greg: is the node that terminates the traffic the decap node?
Barak: yes. It is not the original packet. You don't drop data packets.
Greg: so why drop the packet?
Barak: if you want to measure performance then use the same QoS as the
original packets, but if it is not performance then you do not
necessararily use same QoS.
Greg: in BFD you use the same QoS as the original data.
Barak: BFD is very different.
Greg: the document should describe what is the use case.
Barak: it is already described.
Frank: the active flag gives protocols the ability to indicate that this is
active OAM. We are not specifying what the active OAM is. any active OAM
mechanism can consider using it. There may be various mechanisms which use
this. For example BFD.
Greg: what if you generate a performance measurement message?
Barak: if you encap IOAM, then you are an encapsulating node.
Tal: right, the encapsulating node generates the active OAM packet.
Barak: if a host sends a test packet through the encap node, then it can
tell the encap node that it is active.
Barak: one of the use cases was cloning a data packet, and setting the
active flag in it.
Greg: which document describes this?
Greg: the current draft does not build the case for introducing this flag.
Shwetha: what is the concern?
Greg: what is the reason for introducing more active OAM mechanics. You can
already use active OAM. There is no reason to differentiate data packet
from test packet.
Barak: if you want to measure the performance of data packets, how do you
do it today?
Greg: you have tools today for active OAM.
Barak: some people found this technique useful.
Frank: Greg says there is a set of active OAM mechanisms. You can push IOAM
onto them. why do you need the additional bit?
Barak: Jay suggested it, and Mickey followed up. IOAM is expensive to
implement. This mechanism helps make it less expensive.
Frakn: it helps hardware, and chip vendors see it as helpful.
Greg: please phrase the way it is used by the implementation, and the fact
it helps the hardware.
Shwetha: if the flag is set, you avoid the matching for decap, right?
Barak: the classification will happen anyway.
Frank: emphasize how it helps the implementer.
Tal: to summarize, what we want to do is clarify in the flag draft that the
active flag helps implementations and simplifies the behavoir of the decap
node.

New flag discussion
===================
Ramesh: there was a proposal to add a flag for a case where there is an
incoming VXLAN, and you switch to VXLAN-GPE in order to push IOAM. The bit
says whether you switch it back to VXLAN.
Frank: not sure we want a flag for that.
Tal: right, it sounds like a problem at a different layer, not in the IOAM
layer.
Ramesh: we should continue the discussion when Mickey is present, since he
is the one who suggested it.


Data draft discussion
=====================
Greg: some objects seem to be presented in two different sizes - long /
short format. What is the reason for two different formats?
Shwetha: it is an optimization issue, of whether you need a long or short
field.
Greg: what is the case for the long ID and the short ID? Why not one
format? It needs to be revisited, or sufficiently explained.
Shwetha: do you think we need a draft that clarifies how to use the fields?
Greg: what is the reason for having two different sizes? Is it historical?
There needs to be a strong case.
Frank: we need to go and clarify it. Probably in the deployment draft and
not the data draft. It is not an issue of whether I use long or short. You
can actually use one of them or a combination of the two, allowing you an
even longer space.
Greg: then why not variable length?
Frank: variable length is much more difficult for hardware.
Greg: it is already very complex to have so many combinations of different
options.
Frank: that is why we have a bit field, which is more friendly than a TLV.
We have opaque state field which allows you to do whatever you want.
Greg: it is not a question of why fixed format is more friendly than TLV.
People will be implementing it. Why not simplify?
Frank: that is what we tried to do by fixed fields.
Al: multiple lengths for timestamps have been around for a long time.
Greg: NTP and PTP have different formats, but test protocols have a very
specific timestamp length.
Tal: bottom line, how should we improve the document?
Greg: either use a TLV, or make it the largest size, and explain how it can
be used to represent the short fields as a subset of the bits.
Shwetha: that would not work.
Rakesh: you are using extra space for that. That will not work.
Frank: we can raise it to the working group.
Barak: many operators complain about the overhead.
Greg: how about you keep the small fields, and allow the long format using
a TLV?
Barak: complicated.
Frank: we can discuss it.
Shwetha: do we really want to do that?
Frank: it is a fair question, to consider dropping the long format.
Barak: this was brought up in the past. We have a TLV, but it cannot be
controlled from the edge. How can you use the TLV from the edge?



============================
Tuesday, November 19th, 2019
12:00
Meeting Minutes
============================

Attendees
=========
Tal Mizrahi, Barak Gafni, Greg Mirsky, Shwetha Bhandari, Justin Iurman,
Ramesh Sivakolundu, Frank Brockners, Roni Even.


IPv6 Option draft discussion
============================
Tal: any open issues on the data draft?
Frank: the IPv6 open issue on the IPPM meeting: the text in the draft was
intentional in order to comply with RFC 8200. If you receive a packet that
has an IOAM extended header on an interface that is not supposed to receive
it, it must be dropped.
Ramesh: so you are forcing a configuration on the port?
Frank: right. That is implied by RFC 8200. Intended to prevent information
leaking to the global internet.
Barak: on the ingress or egress interface?
Frank: not sure about the exact text.
Greg: be conservative in what you send and liberal in what you receive. If
you send something that is not allowed that breaks the rules.
Barak: the question is if the configuration is an ingress property, an
egress property, or both.
Justin: in the kernel there is a distinction between ingress and egress.
Barak: LACP is a unidirectional configuration.
Frank: after checking the document: the current phrasing in the draft talks
about ingress behavior.
Ramesh: so the comment from 6MAN was that it should be dropped if received
through an interface that is not enabled for IOAM?
Frank: right.
Greg: so if a router is not IOAM capable then it would not be able to know
what IOAM is.
Frank: right.
Tal: so today if you receive an unknown extension header you drop the
packet?
Frank: no.
Greg: RFC 8200 is complicated.
Roni: the right way is to have correct behavior even if you do not know
what IOAM is.
Greg: not sending it silently is the right way.
Shwetha: we want for nodes that do not know IOAM to forward the packet, but
nodes that are IOAM aware but not enabled for IOAM should drop.
Frank: right, that is what the text says.
Tal: to summarize, at this point we are keeping the text as-is?
Frank: subject to comments we may receive on the mailing list.


IOAM Profile Draft
==================
Tal: the IOAM profile draft defines the concept of a profile, which is a
specific use case that uses a specific subset of IOAM features. An IOAM
profile may be defined with the help of assigning a specific allocation to
the IOAM Yang model. Question: do we need a specific Yang model for the
IOAM profile draft, or can we just use the IOAM Yang model?
Frank: we can change the existing yang model if necessary.
Tal: a profile can be defined by a document, and assigning a value to the
IOAM Yang model helps to specify the profile. However, some aspects cannot
be defining using the IOAM Yang model. For example, a profile may define
specific semantics for the Opaque State Snapshot, and such semantics cannot
be defined by simply assigning a value in a Yang model.
Frank: does this limit the generality of the Opaque field?
Barak: somewhere it needs to be specified.
Greg: the YANG model is read / write, so let's not limit its functionality.
Tal: right, but the Yang model is used for configuration, not for reading
the IOAM data itself. I suggest to use the existing IOAM Yang model.
Additional functionality like opaque fields should be defined in the
respective document that specifies the profile.
Greg: the semantics also need to be configured. How to do that?
Tal: are you saying that the profile defines the opaque field and also
amendment to Yang model that allows to configure it?
Greg: yes.
Shwetha: only the schema?
Tal: yes.
Barak: are we maintaining the Yang model?
Frank: yes.
Ramesh: the model needs work.
Tal: to summarize, does it make sense that we use the existing IOAM Yang
model and in addition a profile may augment the model in order to define
functionality that is specific to that profile?
Frank: yes, let's not define something new for each profile.


Multicast discussion
====================
Barak: someone said the current draft supports only unicast.
Greg: not me. People do not think about the implications of collecting the
data on data packets, since packets are replicated in multicast.
Barak: we need to think about whether the current draft supports multicast.
In my opinion yes. We need to think how we can optimize multicast.
Greg: the DEX proposal is applicable to multicast. I do not think that DEX
has this problem. I think that in DEX there is an issue about correlation.
Barak: do you agree that the current IOAM supports multicast, and that we
may need another draft for multicast?
Greg: DEX is okay with multicast. There is a proposal of how to resolve the
correlation problem. Regarding hop-by-hop option, this needs more
discussion about how it is affected by multicast.
No objections.