[ippm] Benjamin Kaduk's Discuss on draft-ietf-ippm-initial-registry-14: (with DISCUSS and COMMENT)

[Benjamin Kaduk via Datatracker <[e-mail]>]

Benjamin Kaduk has entered the following ballot position for
draft-ietf-ippm-initial-registry-14: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-ippm-initial-registry/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Quite minor points, really, but they do need to be resolved before
publication.  (Slightly more details/locations in the COMMENT section.)

We don't actually provide a definition (whether directly or by
reference) for the "classic calculation for standard deviation of a
population".

We don't actually provide a definition (whether directly or by
reference) for what the "time_offset" calibration value records.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Section 2

   This document defines the initial set of Performance Metrics Registry
   entries, for which IETF approval (following development in the IP
   Performance Metrics (IPPM) Working Group) will satisfy the
   requirement for Expert Review.  Most are Active Performance Metrics,

That's not really how Expert Review works, though as is being discussed
in other ballot threads, we have other options for getting these registered.

Section 4

   Note: Each Registry entry only produces a "raw" output or a
   statistical summary.  To describe both "raw" and one or more
   statistics efficiently, the Identifier, Name, and Output Categories
   can be split and a single section can specify two or more closely-
   related metrics.  This section specifies two Registry entries with
   many common columns.  See Section 7 for an example specifying
   multiple Registry entries with many common columns.

I'm not sure I understand the intended scope/audience of this remark.
It is discussing something that we are doing for these registrations or
something that might be done in the future?

Section 4.2.2

   o  UDP Payload

      *  total of 100 bytes

In the vein of "clear and reproducible metric definition", should we say
something about the content of the payload (even if it's just "the
content is arbitrary")?  Per Section 4.3.1 we could even note that a
sequence number could be used as the payload.
(I won't mention a similar consideration for the later metrics.)

   o  Tmax: a loss threshold waiting time

      *  3.0, expressed in units of seconds, as a positive value of type
         decimal64 with fraction digits = 4 (see section 9.3 of
         [RFC6020]) and with resolution of 0.0001 seconds (0.1 ms), with
         lossless conversion to/from the 32-bit NTP timestamp as per
         section 6 of [RFC5905].

Am I reading this right that the Tmax is a fixed parameter with value
3.0 seconds?  If so, I'm confused why the extra specification on its
representation is needed (and why we write only "3.0" and not "3.0000"
to get the 4 fraction digits, though per RFC 6020 the fraction digits is
just a maximum).
(I won't mention a similar consideration for the later metrics.)

Section 4.3.1

   The calculations on the delay (RTT) SHALL be performed on the
   conditional distribution, conditioned on successful packet arrival
   within Tmax.  Also, when all packet delays are stored, the process
   which calculates the RTT value MAY enforce the Tmax threshold on
   stored values before calculations.  See section 4.1 of [RFC3393] for

Why is this a MAY and not a MUST?  Wouldn't it make the results not
necessarily reproducible/transferrable?
(I won't mention a similar consideration for the later metrics.)

Section 4.3.2

   dT the duration of the interval for allowed sample start times, with
      value 1.0, expressed in units of seconds, as a positive value of
      type decimal64 with fraction digits = 4 (see section 9.3 of
      [RFC6020]) and with resolution of 0.0001 seconds (0.1 ms).

[1.0 does not have 4 fraction digits; though we did use 0.0200 for incT]

Section 4.3.5

There are potentially privacy considerations (albeit likely to be
minimal) with requiring the reporting of Src and Dst addresses as part
of the measurement results.

Section 4.4.2

   95Percentile  The time value of the result is expressed in units of
      seconds, as a positive value of type decimal64 with fraction
      digits = 9 (see section 9.3 of [RFC6020]) with resolution of
      0.000000001 seconds (1.0 ns), and with lossless conversion to/from
      the 64-bit NTP timestamp as

Is there a missing end of the sentence?

Section 4.4.4

   When a measurement controller requests a calibration measurement, the
   loopback is applied and the result is output in the same format as a
   normal measurement with additional indication that it is a
   calibration result.

If the goal of the calibration is intended in part to quantify the
random errors of a measurement, shouldn't we also report something akin
to a variance or standard distribution, as opposed to just the same
summary results that we would get from a non-calibration measurement?
My understanding of what this says to do will only give you the
"average offset" due to the random+systematic error (for the appropriate
definition of "average") but not give you a sense for whether the
effective precision is reduced by those random errors.
(I won't mention a similar consideration for the later metrics.)

Section 5.4.4

Does this really give me a clear definition of what the time_offset is
measuring (as opposed to just how to represent it)?
(I won't mention a similar consideration for the later metrics.)

Section 6.3.1

   DNS Messages bearing Queries provide for random ID Numbers in the
   Identification header field, so more than one query may be launched
   while a previous request is outstanding when the ID Number is used.
   Therefore, the ID Number MUST be retained at the Src or included with
   each response packet to disambiguate packet reordering if it occurs.

Is this an "or" or an "and"?  Also, the DNS protocol is pretty well
specified about this already, right, so we're just noting the properties
that is has?

   In addition to operations described in [RFC2681], the Src MUST parse
   the DNS headers of the reply and prepare the information for
   subsequent reporting as a measured result, along with the Round-Trip
   Delay.

Since we only use the Rcode, maybe we could be more specific than "the
information"?

Section 6.3.2

   Section 11.1.3 of RFC 2681 [RFC2330] provides three methods to
   generate Poisson sampling intervals.  The reciprocal of lambda is the
   average packet rate, thus the Run-time Parameter is Reciprocal_lambda
   = 1/lambda, in seconds.

Wouldn't something measured in seconds be an inter-packet interval, not
a "packet rate"?  (Hmm, § 7.3.2 calls it the "average packet spacing".)

Section 6.3.5

   Trunc  Upper limit on Poisson distribution expressed in units of
      seconds, as a positive value of type decimal64 with fraction
      digits = 4 (see section 9.3 of [RFC6020]) with resolution of
      0.0001 seconds (0.1 ms), and with lossless conversion to/from the
      32-bit NTP timestamp as per section 6 of [RFC5905] (values above
      this limit will be clipped and set to the limit value). (if fixed,
      Trunc = 30.0000 seconds.)

For a registered matric, a parameter is either Fixed or Run-time; there
is no option.  This last parenthetical needs to be removed.

   ID The 16-bit identifier assigned by the program that generates the
      query, and which must vary in successive queries, see
      Section 4.1.1 of [RFC1035].  This identifier is copied into the
      corresponding reply and can be used by the requester (Src) to
      match-up replies to outstanding queries.

If it must vary in successive queries (which are still part of the same
Stream), that's not a single Run-time parameter anymore, is it?

   QTYPE  The Query Type, which will correspond to the IP address family
      of the query (decimal 1 for IPv4 or 28 for IPv6, formatted as a
      uint16, as per section 9.2 of [RFC6020].

Does this implicitly exclude non-IP-address queries?

Section 7.2.2

Do we need to report or specify which TWAMP security features are in
use, in case that has an effect on processing times and the
corresponding measurements?
(I won't mention a similar consideration for the later metrics.)

Section 7.3.1

   The reference method requires some way to distinguish between
   different packets in a stream to establish correspondence between
   sending times and receiving times for each successfully-arriving
   packet.  Sequence numbers or other send-order identification MUST be
   retained at the Src or included with each packet to disambiguate
   packet reordering if it occurs.

   Since a standard measurement protocol is employed [RFC5357], then the
   measurement process will determine the sequence numbers or timestamps
   applied to test packets after the Fixed and Runtime parameters are
   passed to that process.  The measurement protocol dictates the format
   of sequence numbers and time-stamps conveyed in the TWAMP-Test packet
   payload.

I'm not sure whether it makes sense to keep the end of the first
paragraph given that the second paragraph covers the specifics of how
the requirement is satisfied.
(I won't mention a similar consideration for the later metrics.)

Section 7.4.2.5

   See section 4.3.2 of [RFC6049] for a closely related method for
   calculating this statistic, and 4.3.3 of [RFC6049].  The formula is
   the classic calculation for standard deviation of a population.

Please provide the actual formula, whether directly or by reference.  A
"closely related method" implies that it is not the actual method to
use, leaving the actual method to use underspecified here.

Section 9

Interesting to see that we don't include 95thpercentile or StdDev for
ICMP, though this is of course not actually problematic.

   All column entries beside the ID, Name, Description, and Output
   Reference Method categories are the same, thus this section proposes
   two closely-related registry entries.  As a result, IANA is also
   asked to assign four corresponding URLs to each Named Metric.

nit: s/four//

Section 9.2.2

   o  ICMP Payload

      *  total of 32 bytes of random info

Randomly selected when -- per-test?  per-packet?
I note that this is the "fixed parameters" section, and the above seems
to be anything but *fixed*.

Section 10

   All column entries beside the ID, Name, Description, and Output
   Reference Method categories are the same, thus this section proposes
   four closely-related registry entries.  As a result, IANA is also
   asked to assign four corresponding URLs to each Named Metric.

nit: s/four//

Section 10.1.2

nit: s/opportuinty/opportunity/

Section 10.2.1


   Traffic filters reduce the packet stream at the OP to a Qualified
   bidirectional flow packets.

nit: singular/plural mistmatch "a"/"packets"

   For a real number, RTD_fwd, >> the Round-trip Delay in the Forward
   direction from OP to host B at time T' is RTD_fwd << REQUIRES that OP
   observed a Qualified Packet to host B at wire-time T', that host B
   received that packet and sent a Corresponding Packet back to host A,
   and OP observed the Corresponding Packet at wire-time T' + RTD_fwd.

I'm either failing to parse this or confused by the notation (or both).
Are the doubled angle brackets intended to enclose a parenthetical
remark defining RTD_fwd?  (If so, why would traditional parentheses not
work?)

Section 11

Please mention the potential privacy considerations for observed
traffic, particularly for passive metrics.

An attacker that knows that its TCP connection is being measured can
modify its behavior to skew the measurement results.

   Security.  Each referenced Metric contains a Security Considerations
   section.

I do not see any such sections.

Section 14.2

Shouldn't RFC 5481 be normative, given that in Section 5.3.1 we see that
"this metric entry requires implementation of the PDV form defined in
section 4.2 of [RFC5481]"?