Re: [Bier] AD Review of draft-ietf-bier-pmmm-oam-05

[Alvaro Retana <[e-mail]>]

On June 29, 2019 at 5:22:37 PM, Greg Mirsky ([e-mail]) wrote:

Greg:

Hi!

many thanks for your review, detailed questions, and the most helpful
suggestions. I've prepared the updated new version to address the nits,
minor and major issues you've pointed out. Please find my answers, notes
in-lined tagged GIM>>. We, the authors, will discuss your other comments
and respond to them soon.

As I mentioned before, I will be returning the document to the WG for
additional work.  Please submit an update in line with your responses — I
think there are improvements, but the large underlying issues remain, which
is what I want the WG to discuss.  I will return the document as soon as
you submit the update.

I have a couple of comments inline.

Thanks!

Alvaro.


...

(1) This document uses the PNPM method described in rfc8321, which makes
> that RFC a required Normative reference because it "must be read to
> understand or implement the technology" [1].  Note that some of my comments
> below are precisely about being explicit with the use of PNPM; the text
> talks about the marking method in general...
>
GIM>> Moved to the Normative section of referenced documents.

That’s a required first step…but the question of this document’s status is
the bigger one:

(2) The use of PNPM results then in the fact that this document cannot be
> on the Standards Track because rfc8321 is Experimental.  In general,
> downward references are possible, but I don't think this is one of those
> cases.  The Shepherd writeup for rfc8321 [2] states that "the measurement
> utility of this extension still is to be demonstrated at a variety of
> scales in a plurality of network conditions."  As far as I can tell, that
> hasn't been demonstrated, nor specific information about the completion of
> the experiment was included in the RFC text.  I didn't see the topic of the
> document status discussed in the WG -- nor am I aware of discussions about
> the maturity of rfc8321 in the ippm WG.  The result is then that this
> document should be either Informational or Experimental.
>
> (3) Before digging further into the status of rfc8321, I want to ask the
> question of the applicability of PNPM to multicast traffic, is it?  On one
> hand, I see that rfc8321 reports that the "methodology has been used
> experimentally in Telecom Italia's network and is applied to
> multicast...".  On the other hand, draft-ietf-ippm-multipoint-alt-mark [*]
> starts by saying that rfc8321 "can be applied only to point-to-point
> flows".  I think that we could stretch the use of PNPM to monitor
> (referring to Figure 2) both A-C-G and A-C-F using a single set of markings
> at A...but that piece of the methodology is not specified in the document.
>
GIM>> I think that the characterization of the method described in RFC 8321
in draft-ietf-ippm-multipoint-alt-mark must be revised. The multipoint
draft discusses methods needed to use the Alternate Marking method in cases
with multiple marking nodes, i.e., mp2p and mp2mp use cases. RFC 8321
addressed use case with a single marking node in the network and such use
cases fir with p2p and p2mp scenarios.

It would be a good idea then to clarify those scenarios here, with respect
to the BIER application…. I couldn’t find an explicit discussion of those
scenarios in rfc8321: I searched for p2mp, p2p, multipoint, multi-point…


(4) Finally, what is the relationship between this document and
> draft-ietf-bier-oam-requirements?  How does this document address the
> requirements?  Why isn't draft-ietf-bier-oam-requirements even mentioned?
>
GIM>> Added reference to specific requirements in the Introduction section:
   The method, called Packet Network Performance Monitoring
   (PNPM), can be used to measure packet loss, latency, and jitter on
   live traffic complies with requirements #5 and #12 listed in
   [I-D.ietf-bier-oam-requirements].

Ok…but I-D.ietf-bier-oam-requirements lists 14 total requirements, many of
them with “MUST”.  What about them?


...

[major] It's not clear to me whether PNPM is known as *the* Marking Method,
> or if it is simply *a* marking method.  Please clarify.  I note that the
> later mentions to "marking method" are all in lower case, which seem to
> imply something generic -- if referring to PNPM, it would be better to do
> it explicitly.
>
GIM>> Changed to "a marking method". Removed MM from the Terminology
section.

The text in general talks about “marking method”, not PNPM…it would be
better if the Normative sections talked about PNPM and not just a marking
method.


...

[major] "...if the operator initially monitors A-C-G and A-B-D segments he
> may enable measurements on segments C-F and B-E at any time."  How?  A
> similar question was asked in the Shepherd's review, and the answer
> included this: "the AltMark domain may be arbitrary and not identical to
> the BIER domain. But from operational PoV, I believe, it is useful to apply
> AltMark at BFIR and then clear them by removing BIER encapsulation at
> BFERs." [3]  However, the document doesn't have any type of discussion
> related to the existence of multiple types of domains, or their
> congruence...much less operational guidance.  Please be explicit about the
> potential different domains, their relationship to the specification in
> rfc8296 and provide operational guidance in an Operational Considerations
> section.
>
GIM>> Added the Operational Consideration section.

Assuming that A is the BFIR, the new section doesn’t explain the
measurement of C-F, for example.

...

[major] "this method is sensitive to packet loss and re-ordering of
> packets"  It will be important to point at the Considerations section from
> rfc8321.  It would be ideal to do so in the Operational Considerations
> section.
>
GIM>> Appreciate your feedback on the new section in the updated version of
the draft.

I think of an Operational Considerations section as one that gives
operators guidance on the use of the feature.  In this case, pointing at
rfc8321 helps provide some background, but it doesn’t help the operator in
actual settings like deciding the length of time, or number of packets, to
be used in each monitored flow.


>
> 203   o  Average Packet Delay calculation: an average delay is calculated
> 204      by considering the average arrival time of the packets within a
> 205      single block.  A BFR may collect timestamps for each packet
> 206      received within a single block.  Average of the timestamp is the
> 207      sum of all the timestamps divided by the total number of packets
> 208      received.  Then the difference between averages calculated at two
> 209      monitoring points is the average packet delay on that segment.
> 210      This method is robust to out of order packets and also to packet
> 211      loss (only a small error is introduced).  This method only
> 212      provides a single metric for the duration of the block and it
> 213      doesn't give the minimum and maximum delay values.  This
> 214      limitation could be overcome by reducing the duration of the block
> 215      by means of a highly optimized implementation of the method.
>
> [minor] "Then the difference between averages calculated at two monitoring
> points is the average packet delay on that segment."  Maybe my math is
> rusty, but I would have thought the average delay to be the average of the
> averages, not the difference.  ??
>
GIM>> I think that the text needs improvement to clarify the math used here:
      Then the difference between the average packet arrival
      time calculated for the downstream monitoring point and the same
      metric but calculated at the upstream monitoring point is the
      average packet delay on the segment between these two points.

>
> [minor] "only a small error is introduced"  This seems to be a statement
> of faith: no proof or justification.
>
GIM>> The new text to explain that conclusion added:
      This method is robust to out of order packets and also to packet
      loss on the segment between the measurement points (packet loss
      may cause a minor loss of accuracy in the calculated metric
      because the number of packets used is different at each
      measurement point).

This is true but only as the BIER traffic is “forked off”.


...

240 5.  IANA Considerations
>

> 242   This document requests IANA to register format of the OAM field of
> 243   BIER Header as the following:
>
> [major] §3 calls these bits L and D.
>
GIM>> Missed to update Section 3 to 'S' and 'D'. Should be fixed now.

>
> [major] If you're assigning values to all the bits in the field, then a
> registry is not needed.
>
> [major] If you want to set up a registry to avoid others using the bits in
> different ways, then that requires an Update to rfc8296.  The way I read
> rfc8296 is that there could be multiple ways of using the field.  I also
> didn't see this type of discussion in the WG archive.
>
> 245   +--------------+---------+--------------------------+---------------+
> 246   | Bit Position | Marking | Description              | Reference     |
> 247   +--------------+---------+--------------------------+---------------+
> 248   |      0       |    S    | Single Mark Measurement  | This document |
> 249   |      1       |    D    | Double Mark Measurement  | This document |
> 250   +--------------+---------+--------------------------+---------------+
>
> 252                     Table 1: OAM field of BIER Header
>
> [major] §3 uses a different description, which seems inline with §4.
> Also, the use of a single bit doesn't indicate (according to §4) the type
> of measurement done.
>
GIM>> I recall that the naming of the flags was discussed and there was a
suggestion to use more generic names than Loss and Delay because other
performance metrics can be measured using this method.

My point was that the name doesn’t reflect the use for what is described in
this document.  In the Double-Marking case, for example, both flags are
used.

As I mentioned above, if this document wants to define the use of the bits
in general, then an Update to rfc8296 will be needed.  Personally, I don’t
think that is the best direction…specially since the a priori configuration
(and not in-line signaling) is used for the nodes to know which OAM method
is being used.  But that is just my personal opinion…the WG can decide
something different.

GIM>> Also, a more compact marking method has been proposed in
draft-mizrahi-ippm-compact-alternate-marking
<https://datatracker.ietf.org/doc/draft-mizrahi-ippm-compact-alternate-marking/>.
This method allows the measurement of packet loss and packet delay using
one-bit long field.

Ok…but that draft is not mentioned in this document?  Are you suggesting
that the compact marking method could be used instead (?) of what is
defined in rfc8321?  This obviously needs more WG discussion as well.