Re: [pm-dir] Performance metrics doctors generated email

["MORTON JR., ALFRED (AL)" <[e-mail]>]

Hi Paul,

Our generally accepted mode of operation is in the message
appended below.  We need to make a few changes now that the
script is finding more drafts than there are Directorate members.

We could choose to expand our membership, that's a question I 
asked the Directorate a few weeks ago, but with no feedback there
was no action.
	When the new list was announced, several people tried to join.
	We should probably have a call for new members soon, 
	if the current Directorate members agree.  Also, if anyone would 
	like to retire from the Performance Directorate (for example,
	if you didn't find time to perform reviews in 2012, then
	you might consider the question in that light), please let me know.

I agree that we need a more accessible and now, more dynamic means of 
organization.  A Wiki will work, as long as everyone has access to it
and can update draft status as they complete reviews, etc.

Once we handle the initial rush of drafts from the 
script, I would suggest that the script keep last week's list and
highlight the new arrivals to facilitate assignment, rather than 
leave this step for manual checking.

regards,
Al
PM Dir admin

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
At 09:47 AM 1/16/2012, Al Morton wrote:
As this new year goes into full swing, I'd like to offer some
thoughts on how we might operate as a Directorate and
hear from others.

As our web page says: "RFC 6390 is used as a reference for the technical 
and process issues." Have your copy handy when reviewing a draft.
A lot of interactions may be streamlined by pointing the authors to 6390,
and mentioning that it is a BCP.

I feel that our reviews should be *early* in the draft development
process, and that no performance-related draft should start IETF Last Call 
without previous review against 6390, unless it truly slipped by us
and other folks who know about the Directorate.  

There are two ways we can collect drafts for review:

- WG Chairs and ADs can/should suggest drafts to us

- Directorate Members will see drafts, hear about drafts with performance
work, etc.  We are the eyes and ears of the Directorate. We all have friends
working in other areas of IETF, let them know what we're looking for.

I suggest to split the early review into two categories:

1. Quick Scan - If you find a draft, do a quick scan or suggest that
others take a look with a message to pmol-list.  Finding a draft does
not mean that you will be designated to do a review (see below).

2. WG Candidate/Chartered Item - When a draft appears to have enough value
and interest to become a WG item, or the WG Chairs involved recommend a 
Performance Metrics Directorate review on their own, then we do a more
complete review against 6390.  

If these categories are useful, then I'd like to suggest that *two* 
reviewers are assigned to WG Candidate/Chartered drafts and they follow
the draft after early review. This may mean looking at the draft again
in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers
while we're all new at this, and to help ensure that one is still around
years later when Last Calls are done.

All drafts identified, quick scans, and reviews should be sent to the
pmol-list and the authors (and possibly the relevant WG chairs)

let us all know what you think,
Al

> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf
> Of Paul Aitken
> Sent: Monday, February 04, 2013 7:50 AM
> To: Benoit Claise
> Cc: [e-mail]
> Subject: Re: [pm-dir] Performance metrics doctors generated email
> 
> Benoit,
> 
> > Yes, all these questions are important.
> > Yes a WIKI is the right solution IMHO.
> 
> Great. Where can it be hosted?
> 
> eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting
> place? Or is there a pm-dir page?
> 
> P.
> 
> >> Benoit, All,
> >>
> >> How is pm-dir tracking these drafts? ie, how do we know whether
> >> someone is currently reviewing, or has already reviewed, each of
> >> these drafts?
> >>
> >> How are reviewers selected? By relevance, skill, luck, or misfortune?
> >>
> >> Also, it'd be useful to know whether the previous version was already
> >> reviewed, and what the feedback was:
> >>
> >>     * issues were raised, recommendations were made, changes were
> >> expected -> some re-review should be done
> >>
> >>     * no issues were raised -> a quick review of the delta may be
> >> sufficient, to check for any new issues
> >>
> >>     * previous version wasn't reviewed -> a thorough review may be
> >> required.
> >>
> >>
> >> I'm thinking it'd be useful to have a pm-dir wiki listing each of the
> >> drafts below. eg, one page per draft, with one section per version,
> >> containing reviewer comments pertinent to that version, even if the
> >> comments simply say, "reviewed, no issues".
> >>
> >> Hopefully I missed something, and most (if not all?) of this is
> >> already in place? :-)
> >>
> >> Thanks,
> >> P.
> >>
> >>
> >> On 04/02/13 11:53, Benoit Claise wrote:
> >>> Dear all,
> >>>
> >>> This is an automatically generated email.
> >>> It lists the IETF internet-drafts that reference the PMOL RFC 6390,
> >>> as a normative or informative reference.
> >>> It also lists all the IETF internet-drafts that contain "performance
> >>> metric".
> >>>
> >>> Regards, Benoit
> >>>
> >>> ===========================================================
> >>>
> >>> Normative References
> >>> --------------------
> >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
> >>>      Informative References
> >>> ----------------------
> >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
> >>> - ID Tracker state <Publication Requested>
> >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
> >>> - ID Tracker state <Publication Requested>
> >>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
> >>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
> >>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
> >>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
> >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
> >>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
> >>>
> >>> drafts containing performance metric
> >>> ------------------------------------
> >>> draft-ietf-alto-deployments-05                    Active
> >>> draft-ietf-alto-protocol-13                       In IESG processing
> >>> - ID Tracker state <AD Evaluation::Revised ID Needed>
> >>> draft-ietf-ippm-rate-problem-02                   Active
> >>> draft-ietf-ippm-testplan-rfc2680-01               Active
> >>> draft-ietf-karp-threats-reqs-07                   In IESG processing
> >>> - ID Tracker state <RFC Ed Queue>
> >>> draft-ietf-manet-olsrv2-mib-05                    Active
> >>> draft-ietf-manet-smf-mib-06                       Active
> >>> draft-ietf-mmusic-media-loopback-27               In IESG processing
> >>> - ID Tracker state <RFC Ed Queue>
> >>> draft-ietf-nvo3-framework-01                      Active
> >>> draft-ietf-opsawg-oam-overview-08                 In IESG processing
> >>> - ID Tracker state <Waiting for AD Go-Ahead::Revised ID Needed>
> >>> draft-ietf-rtcweb-rtp-usage-05                    Active
> >>> draft-ietf-rtgwg-cl-framework-02                  Active
> >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
> >>> - ID Tracker state <Publication Requested>
> >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
> >>> - ID Tracker state <Publication Requested>
> >>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
> >>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
> >>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
> >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
> >>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
> >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
> >>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
> >>> draft-ietf-xrblock-rtcp-xr-summary-stat-06        In IESG processing
> >>> - ID Tracker state <Waiting for AD Go-Ahead>
> >>> draft-ietf-xrblock-rtcp-xr-synchronization-02     Active
> >>> _______________________________________________
> >>> pm-dir mailing list
> >>> [e-mail]
> >>> https://www.ietf.org/mailman/listinfo/pm-dir
> >>
> >> _______________________________________________
> >> pm-dir mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/pm-dir
> >>
> >>
> 
> _______________________________________________
> pm-dir mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/pm-dir