Re: [pm-dir] Performance metrics doctors generated email

[Benoit Claise <[e-mail]>]

Al, All,
> Hi Paul,
>
> Our generally accepted mode of operation is in the message
> appended below.  We need to make a few changes now that the
> script is finding more drafts than there are Directorate members.
>
> We could choose to expand our membership, that's a question I
> asked the Directorate a few weeks ago, but with no feedback there
> was no action.
> 	When the new list was announced, several people tried to join.
> 	We should probably have a call for new members soon,
> 	if the current Directorate members agree.  Also, if anyone would
> 	like to retire from the Performance Directorate (for example,
> 	if you didn't find time to perform reviews in 2012, then
> 	you might consider the question in that light), please let me know.
>
> I agree that we need a more accessible and now, more dynamic means of
> organization.  A Wiki will work, as long as everyone has access to it
> and can update draft status as they complete reviews, etc.
Agreed.
>
> Once we handle the initial rush of drafts from the
> script, I would suggest that the script keep last week's list and
> highlight the new arrivals to facilitate assignment, rather than
> leave this step for manual checking.
That would nice, but we would need a kind of database to keep track of 
the already assigned drafts.
It starts to be a more complex script...

Note: Alan (in the copy list) is improving the script so that the drafts 
that maps a keyword (performance metric in this case) and that have a 
reference to RFC 6390 doesn't appear twice.

Regards, Benoit
>
> regards,
> Al
> PM Dir admin
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> At 09:47 AM 1/16/2012, Al Morton wrote:
> As this new year goes into full swing, I'd like to offer some
> thoughts on how we might operate as a Directorate and
> hear from others.
>
> As our web page says: "RFC 6390 is used as a reference for the technical
> and process issues." Have your copy handy when reviewing a draft.
> A lot of interactions may be streamlined by pointing the authors to 6390,
> and mentioning that it is a BCP.
>
> I feel that our reviews should be *early* in the draft development
> process, and that no performance-related draft should start IETF Last Call
> without previous review against 6390, unless it truly slipped by us
> and other folks who know about the Directorate.
>
> There are two ways we can collect drafts for review:
>
> - WG Chairs and ADs can/should suggest drafts to us
>
> - Directorate Members will see drafts, hear about drafts with performance
> work, etc.  We are the eyes and ears of the Directorate. We all have friends
> working in other areas of IETF, let them know what we're looking for.
>
> I suggest to split the early review into two categories:
>
> 1. Quick Scan - If you find a draft, do a quick scan or suggest that
> others take a look with a message to pmol-list.  Finding a draft does
> not mean that you will be designated to do a review (see below).
>
> 2. WG Candidate/Chartered Item - When a draft appears to have enough value
> and interest to become a WG item, or the WG Chairs involved recommend a
> Performance Metrics Directorate review on their own, then we do a more
> complete review against 6390.
>
> If these categories are useful, then I'd like to suggest that *two*
> reviewers are assigned to WG Candidate/Chartered drafts and they follow
> the draft after early review. This may mean looking at the draft again
> in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers
> while we're all new at this, and to help ensure that one is still around
> years later when Last Calls are done.
>
> All drafts identified, quick scans, and reviews should be sent to the
> pmol-list and the authors (and possibly the relevant WG chairs)
>
> let us all know what you think,
> Al
>
>> -----Original Message-----
>> From: [e-mail] [mailto:[e-mail]] On Behalf
>> Of Paul Aitken
>> Sent: Monday, February 04, 2013 7:50 AM
>> To: Benoit Claise
>> Cc: [e-mail]
>> Subject: Re: [pm-dir] Performance metrics doctors generated email
>>
>> Benoit,
>>
>>> Yes, all these questions are important.
>>> Yes a WIKI is the right solution IMHO.
>> Great. Where can it be hosted?
>>
>> eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting
>> place? Or is there a pm-dir page?
>>
>> P.
>>
>>>> Benoit, All,
>>>>
>>>> How is pm-dir tracking these drafts? ie, how do we know whether
>>>> someone is currently reviewing, or has already reviewed, each of
>>>> these drafts?
>>>>
>>>> How are reviewers selected? By relevance, skill, luck, or misfortune?
>>>>
>>>> Also, it'd be useful to know whether the previous version was already
>>>> reviewed, and what the feedback was:
>>>>
>>>>      * issues were raised, recommendations were made, changes were
>>>> expected -> some re-review should be done
>>>>
>>>>      * no issues were raised -> a quick review of the delta may be
>>>> sufficient, to check for any new issues
>>>>
>>>>      * previous version wasn't reviewed -> a thorough review may be
>>>> required.
>>>>
>>>>
>>>> I'm thinking it'd be useful to have a pm-dir wiki listing each of the
>>>> drafts below. eg, one page per draft, with one section per version,
>>>> containing reviewer comments pertinent to that version, even if the
>>>> comments simply say, "reviewed, no issues".
>>>>
>>>> Hopefully I missed something, and most (if not all?) of this is
>>>> already in place? :-)
>>>>
>>>> Thanks,
>>>> P.
>>>>
>>>>
>>>> On 04/02/13 11:53, Benoit Claise wrote:
>>>>> Dear all,
>>>>>
>>>>> This is an automatically generated email.
>>>>> It lists the IETF internet-drafts that reference the PMOL RFC 6390,
>>>>> as a normative or informative reference.
>>>>> It also lists all the IETF internet-drafts that contain "performance
>>>>> metric".
>>>>>
>>>>> Regards, Benoit
>>>>>
>>>>> ===========================================================
>>>>>
>>>>> Normative References
>>>>> --------------------
>>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>>>>>       Informative References
>>>>> ----------------------
>>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
>>>>> - ID Tracker state <Publication Requested>
>>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
>>>>> - ID Tracker state <Publication Requested>
>>>>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
>>>>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
>>>>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
>>>>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
>>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
>>>>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
>>>>>
>>>>> drafts containing performance metric
>>>>> ------------------------------------
>>>>> draft-ietf-alto-deployments-05                    Active
>>>>> draft-ietf-alto-protocol-13                       In IESG processing
>>>>> - ID Tracker state <AD Evaluation::Revised ID Needed>
>>>>> draft-ietf-ippm-rate-problem-02                   Active
>>>>> draft-ietf-ippm-testplan-rfc2680-01               Active
>>>>> draft-ietf-karp-threats-reqs-07                   In IESG processing
>>>>> - ID Tracker state <RFC Ed Queue>
>>>>> draft-ietf-manet-olsrv2-mib-05                    Active
>>>>> draft-ietf-manet-smf-mib-06                       Active
>>>>> draft-ietf-mmusic-media-loopback-27               In IESG processing
>>>>> - ID Tracker state <RFC Ed Queue>
>>>>> draft-ietf-nvo3-framework-01                      Active
>>>>> draft-ietf-opsawg-oam-overview-08                 In IESG processing
>>>>> - ID Tracker state <Waiting for AD Go-Ahead::Revised ID Needed>
>>>>> draft-ietf-rtcweb-rtp-usage-05                    Active
>>>>> draft-ietf-rtgwg-cl-framework-02                  Active
>>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
>>>>> - ID Tracker state <Publication Requested>
>>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
>>>>> - ID Tracker state <Publication Requested>
>>>>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
>>>>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
>>>>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
>>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>>>>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
>>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
>>>>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
>>>>> draft-ietf-xrblock-rtcp-xr-summary-stat-06        In IESG processing
>>>>> - ID Tracker state <Waiting for AD Go-Ahead>
>>>>> draft-ietf-xrblock-rtcp-xr-synchronization-02     Active
>>>>> _______________________________________________
>>>>> pm-dir mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/pm-dir
>>>> _______________________________________________
>>>> pm-dir mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/pm-dir
>>>>
>>>>
>> _______________________________________________
>> pm-dir mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/pm-dir
>