Re: [ippm] Genart last call review of draft-ietf-ippm-metric-registry-20

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Roni,
thanks again for these proposed changes.
see replies below, marked [acm]

@Alissa - these changes reply to your DISCUSS on places in
the memo where "RFC" appears, that should be generalized
(when using Specification Required policy).

Al

> -----Original Message-----
> From: Roni Even (A) [mailto:[e-mail]]
> Sent: Monday, December 9, 2019 8:10 AM
> To: MORTON, ALFRED C (AL) <[e-mail]>; [e-mail]; last-
> [e-mail]
> Cc: [e-mail]; [e-mail]
> Subject: RE: [ippm] Genart last call review of draft-ietf-ippm-metric-
> registry-20
> 
> Hi Al,
> 
> Here are the proposed changes for the -22 version. See if they make sense
> 
> 
> Section 3 bullet 1 has RFC change (up to and including the publication of
> one or more RFCs or  I-Ds describing it) to (up to and including the
> publication of one or more immutable document such as an RFC)
[acm] 
ok, with s/document/documents/

> 
> Section 7.1.2  change “Spec: RFC number and major section number that
> specifies this” to “Spec: an immutable document, for RFC the RFC number
> and major section number that specifies this”
[acm] 
ok

> 
> 
> Section 7.1.3 change “The URL SHALL reference a target file that is HTML-
> formated and contains URLs to referenced sections of HTML-ized RFCs” to
> “The URL SHALL reference a target file that is HTML-formated and contains
> URLs to referenced sections of HTML-ized reference document”
[acm] 
I went with 
The URL SHALL reference a target file that is HTML-formatted and contains 
URLs to referenced sections of HTML-ized RFCs, or other reference specifications.

> 
> Section 7.1.7 change “a new RFC is published that changes the  registry
> format.”  to “a new document is published that changes the  registry
> format.”
> 
> Section 7.2 change “This category includes columns to prompt all necessary
> details related to the metric definition, including the RFC reference and
> values of input factors, called fixed parameters, which are left open  in
> the RFC but have a particular value defined by the performance  metric.”
> to  “This category includes columns to prompt all necessary details
> related to the metric definition, including the document reference and
> values of input factors, called fixed parameters, which are left open  in
> the document  but have a particular value defined by the performance
> metric.”
> 
> Note that In 7.2.1 talks about immutable document
[acm] 
I used immutable document in 7.2, instead of RFC.
> 
> Section 7.3.1 change “for implementations referring to the RFC text.” to
> “for implementations referring to the immutable document text.”
[acm] 
ok

> 
> Section 8.1 first paragraph change “compliance with other applicable
> Performance Metric-related RFCs,” to “compliance with other applicable
> Performance Metric-related documents,”
[acm] 
I don't think the Performance Metric Experts are in a position
to know about the universe of PM-related documents, just RFCs.
So I left this one as-is. We're talking about Expert Review here.

> 
> In section 10.3 “The "Reference" column will include an RFC number, an
> approved specification designator from another standards body, other
> immutable document, or the contact person.” Remove “or the contact person”
[acm] 
ok

> 
> 
> Section 11.2 change “including the RFC reference” to “including the
> document reference”
[acm] 
ok

> 
> Section 11.3 change “to relevant sections of  the RFC(s) ” to relevant
> sections of  the document(s)”
> 
> 
> Roni
> 
> > -----Original Message-----
> > From: MORTON, ALFRED C (AL) [mailto:[e-mail]]
> > Sent: Thursday, December 05, 2019 4:04 PM
> > To: Roni Even (A); [e-mail]; [e-mail]
> > Cc: [e-mail]; [e-mail]; Mirja
> > Kuehlewind; Alissa Cooper
> > Subject: RE: [ippm] Genart last call review of draft-ietf-ippm-metric-
> registry-
> > 20
> >
> > Hi Roni,
> >
> > Please keep in mind that this is not a global replacement for RFC ->
> > Specification.
> >
> > Some steps in the process apply only to IESG review with IANA review.
> > Others apply more broadly and we only need to change those.
> >
> > Please help by making your recommendation for changes keeping in mind
> > those two categories.
> >
> > thanks,
> > Al
> >
> > > -----Original Message-----
> > > From: Roni Even (A) [mailto:[e-mail]]
> > > Sent: Thursday, December 5, 2019 3:16 AM
> > > To: [e-mail]; [e-mail]; MORTON, ALFRED C (AL)
> > > <[e-mail]>
> > > Cc: [e-mail]; [e-mail]; Mirja
> > > Kuehlewind <[e-mail]>; Alissa Cooper <[e-mail]>
> > > Subject: RE: [ippm] Genart last call review of draft-ietf-ippm-metric-
> > > registry-20
> > >
> > > Hi,
> > > I think that Al addressed most of them, noted the following are still
> > > not clear
> > >
> > > Section 3 bullet 1 has RFC
> > > Section 7.1.2  for "spec:" talks about RFC, should be more general say
> > > if the specification is RFC then ...
> > > Section 7.1.2 talks only about RFC
> > > Section 7.1.3
> > > Section 7.1.7
> > > Section 7.2 (In 7.2.1 talks about immutable document) Section 7.3.1
> > > end of the first paragraph Section 8.1 first paragraph and third
> > > paragraph Section 11.2 Section 11.3 Section 11.5.2  what is the
> > > requestor is it a document or person?
> > >
> > >
> > > New comment:
> > >
> > > In section 10.3 can the reference be a contact person since the policy
> > > is specification required
> > >
> > > Roni Even
> > >
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Alissa Cooper [mailto:[e-mail]]
> > > > Sent: Wednesday, December 04, 2019 8:00 PM
> > > > To: Mirja Kuehlewind
> > > > Cc: Roni Even (A); MORTON, ALFRED C (AL); Roni Even;
> > > > [e-mail];
> > > last-
> > > > [e-mail]; [e-mail]; draft-ietf-ippm-metric-
> > > [e-mail]
> > > > Subject: Re: [ippm] Genart last call review of
> > > > draft-ietf-ippm-metric-
> > > registry-
> > > > 20
> > > >
> > > > Hi Mirja,
> > > >
> > > > > On Dec 4, 2019, at 11:35 AM, Mirja Kuehlewind
> > > > > <[e-mail]>
> > > > wrote:
> > > > >
> > > > > Hi Alissa,
> > > > >
> > > > > Section 10.1 say:
> > > > >
> > > > > Registration Procedure: Specification Required
> > > > >
> > > > > What else do you think is needed?
> > > >
> > > > What I put in my ballot:
> > > >
> > > > "I'm confused about what the registration policy is for metrics in
> > > > the
> > > new
> > > > registry. If it is Specification Required, then the places in the
> > > document that
> > > > assume new metrics are defined in an RFC need to be generalized,
> > > > because Specification Required need not involve any RFC at all.”
> > > >
> > > > Best,
> > > > Alissa
> > > >
> > > > >
> > > > > Mirja
> > > > >
> > > > >
> > > > >
> > > > >> On 4. Dec 2019, at 17:15, Alissa Cooper <[e-mail]>
> wrote:
> > > > >>
> > > > >> Roni, thanks for your review. Al, thanks for your response. I
> > > > >> entered
> > > a
> > > > DISCUSS ballot to get the registration policy clarified.
> > > > >>
> > > > >> Alissa
> > > > >>
> > > > >>
> > > > >>> On Nov 1, 2019, at 11:54 AM, Roni Even (A)
> > > > >>> <[e-mail]>
> > > > wrote:
> > > > >>>
> > > > >>> Hi Al,
> > > > >>> I saw that IANA was consulted during the work.
> > > > >>> I was wondering what will be the actual text that will be
> > > > >>> written in
> > > the
> > > > IANA registry, I expected section 10 to describe it.
> > > > >>>
> > > > >>> Registration Procedure(s)
> > > > >>> Reference
> > > > >>> Note
> > > > >>>
> > > > >>> I am not sure yet what is the Registration Procedure and what
> > > > >>> will be written in the Note
> > > > >>>
> > > > >>> Thanks
> > > > >>> Roni
> > > > >>>
> > > > >>> -----Original Message-----
> > > > >>> From: Gen-art [mailto:[e-mail]] On Behalf Of
> > > > MORTON,
> > > > >>> ALFRED C (AL)
> > > > >>> Sent: Thursday, October 31, 2019 11:52 PM
> > > > >>> To: Roni Even; [e-mail]
> > > > >>> Cc: [e-mail];
> > > > >>> [e-mail]; [e-mail]
> > > > >>> Subject: Re: [Gen-art] Genart last call review of
> > > > >>> draft-ietf-ippm-metric-registry-20
> > > > >>>
> > > > >>> Hi Roni,
> > > > >>> thanks for your comments, please see replies below.
> > > > >>> Al
> > > > >>>
> > > > >>>> -----Original Message-----
> > > > >>>> From: Roni Even via Datatracker [mailto:[e-mail]]
> > > > >>>> Sent: Tuesday, October 29, 2019 4:25 AM
> > > > >>>> To: [e-mail]
> > > > >>>> Cc: [e-mail]; [e-mail]; draft-ietf-ippm-metric-
> > > > >>>> [e-mail]
> > > > >>>> Subject: Genart last call review of
> > > > >>>> draft-ietf-ippm-metric-registry-20
> > > > >>>>
> > > > >>>> Reviewer: Roni Even
> > > > >>>> Review result: Almost Ready
> > > > >>>>
> > > > >>>> I am the assigned Gen-ART reviewer for this draft. The General
> > > > >>>> Area Review Team (Gen-ART) reviews all IETF documents being
> > > > >>>> processed
> > > > by
> > > > >>>> the IESG for the IETF Chair.  Please treat these comments just
> > > > >>>> like any other last call comments.
> > > > >>>>
> > > > >>>> For more information, please see the FAQ at
> > > > >>>>
> > > > >>>> <https://urldefense.proofpoint.com/v2/url?u=https-
> > > > >>>> 3A__trac.ietf.org_trac_gen_wiki_GenArtfaq&d=DwICaQ&c=LFYZ-
> > > > >>>>
> > > >
> > o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mLefZkw5Y_ld2AFv2
> > > > msgpz
> > > > >>>> OV5 Z7lZ
> > > > >>>> JkKTdUQf48X15g&s=uUg9ktSDILsslqK-rG4YIc3gMW0n6oCa-
> > > > 7Dk0xtFZRo&e=>.
> > > > >>>>
> > > > >>>> Document: draft-ietf-ippm-metric-registry-??
> > > > >>>> Reviewer: Roni Even
> > > > >>>> Review Date: 2019-10-29
> > > > >>>> IETF LC End Date: 2019-11-06
> > > > >>>> IESG Telechat date: Not scheduled for a telechat
> > > > >>>>
> > > > >>>> Summary:
> > > > >>>> The document is almost ready for publication as a BCP document
> > > > >>>>
> > > > >>>> Major issues:
> > > > >>>>
> > > > >>>> Minor issues:
> > > > >>>> 1. From reading the document it looks to me that the
> > > > >>>> registration policy should be specification required which also
> > > > >>>> requires expert
> > > > review.
> > > > >>> [acm]
> > > > >>> I understand that perspective. In early review with IANA we
> > > > >>> decided
> > > on
> > > > Expert Review partly because two elements of registry entries
> > > > require references to immutable documents, such as standards
> > specifications.
> > > > >>> So the requirement for specifications could be seen as built-in.
> > > > >>> But we may change to Specification Required now, the last IANA
> > > review
> > > > is in-progress.
> > > > >>>
> > > > >>>> 2. My understanding is that for registration a document is
> > > > >>>> required , not necessarily and RFC, but in multiple places in
> > > > >>>> the document ( 7.3, 7.3.1, 8.2 ,...) the text talks about RFC
> and not
> > document.
> > > > >>> [acm]
> > > > >>> Yes, a few of those slipped through, thanks.
> > > > >>>
> > > > >>>> 3. I am not sure if section 6 is needed in the published
> > > > >>>> document
> > > based
> > > > on its content.
> > > > >>> [acm]
> > > > >>> it's fairly easy for new implementers to pick-up an IPPM RFC
> > > > >>> (even a
> > > > STD) and choose parameters that meet their needs. But for the
> > > > additional advantage of measurement comparisons, more context is
> > > > needed. Some may even ask why this registry requires the many
> > > > details. Answer: See section 6.
> > > > >>> A little history is good. Very few have been joining IPPM
> > > > >>> sessions
> > > long
> > > > enough to know this history.
> > > > >>>
> > > > >>>> If it will remain then in 6.1
> > > > >>>> first paragraph the reference should be to section 5 and not to
> > > section
> > > > 6.
> > > > >>> [acm] ok
> > > > >>>
> > > > >>>> 4.
> > > > >>>> In sections 10.2 and 10.3 there are guidance taken from this
> > > document.
> > > > >>>> I think that the for IANA it should say in the registry note
> > > > >>>> that the registration must comply with RFCXXX (this document),
> > > > >>>> I assume that there is no need to repeat all this text in these
> > > > >>>> sections in
> > > the
> > > > registry note.
> > > > >>> [acm]
> > > > >>> I have said on a few occasions that almost the entire memo
> > > > >>> contains
> > > > IANA Considerations. Nevertheless, we wrote and reviewed the memo
> > > > and (then wrote) the separate IANA section with IANA's help.
> > > > >>>
> > > > >>> I have implemented the agreed changes above in the working
> > version.
> > > > >>> Thanks again!
> > > > >>>
> > > > >>>>
> > > > >>>> Nits/editorial comments:
> > > > >>>>
> > > > >>>
> > > > >>> _______________________________________________
> > > > >>> Gen-art mailing list
> > > > >>> [e-mail]
> > > > >>> https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__www.ietf.org_mailman_listinfo_gen-2Dart&d=DwIGaQ&c=LFYZ-
> > > o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-
> > 6zYMI&m=gzua
> > > d-V-
> > >
> > BNukSa9xX1XrkjMj1QZalBXLPIiwWdbDATE&s=vQLx27IlYqGWfgVPigggpoWFf
> > BJX5TJx
> > > z8oU
> > > Pu2NQp0&e=
> > > > >>>
> > > > >>> _______________________________________________
> > > > >>> ippm mailing list
> > > > >>> [e-mail]
> > > > >>> https://urldefense.proofpoint.com/v2/url?u=https-
> > > 3A__www.ietf.org_mailman_listinfo_ippm&d=DwIGaQ&c=LFYZ-
> > > o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-
> > 6zYMI&m=gzua
> > > d-V-
> > >
> > BNukSa9xX1XrkjMj1QZalBXLPIiwWdbDATE&s=HKT1QAvRtO4211JXIR5edFGw
> > 5AS6H94f
> > > 4tFm
> > > saAwWRQ&e=
> > > > >>
> > > > >>
> > > > >