Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06

[Gonzalo Camarillo <[e-mail]>]

Hi Benoit,

thanks for following this up. I have already told Dan, who is this
document's shepherd, to get involved in the discussions so that we can
move forward.

Cheers,

Gonzalo

On 20/02/2013 1:58 AM, Benoit Claise wrote:
> Dear pm-dir,
> 
> [reducing the list to pm-dir and Gonzallo, XRBLOCK AD]
> I filed a DISCUSS on this document.
> See
> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-stat/ballot/#benoit-claise
> for the details.
> We should give the right guidelines to performance metrics related draft
> authors.
> Maybe we should even mention
> http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/#benoit-claise
> in our WIKI?
> 
> Regards, Benoit
>> Dear authors,
>>
>> I'm reviewing version 8 for the IESG call next week.
>>> Benoit,
>>>
>>> Since there is a short term deadline 
>>> (the Last Call has ended on Feb 1) 
>>> I'll provide the PM Dir review.
>>>
>>> Al
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>> Since the draft describes reporting formats and (apparently) does not 
>>> intend to define new metrics, much of 6390 guidance is out of scope.
>> This is the key question.
>> Do you define new metrics?
>> When I see the following (to take just one example):
>>
>>      Burst Loss Rate: 16 bits
>>
>>           The fraction of packets lost during bursts since the beginning of
>>           reception, expressed as a fixed point number with the binary point
>>           at the left edge of the field.  This value is calculated by
>>           dividing Packets Loss in Bursts by Total Packets expected in
>>           Bursts, multiplying the result of the division by 7FFF, with the
>>           maximum value 7FFF, and taking the integer part as follows:
>>
>>           Packets Loss in Bursts / Total Packets expected in Bursts
>>
>>           If the measurement is unavailable, the value 0x8000 MUST be
>>           reported.
>>
>> ... it seems to me that you define new metrics, and that therefore the
>> RFC 6390 template should be applied.
>> What do I miss?
>>
>> Regards, Benoit
>>> However, see the comments on section 4.1.2 below regarding metric definitions.
>>>
>>> Comments:
>>>
>>> Section 1.1 of the Intro says:
>>>
>>>    This draft defines three new block types to augment those defined in
>>>    [RFC3611] for use in a range of RTP applications.
>>>
>>> ok so far, but the next paragraph is hard to decipher, perhaps a simple
>>> numbered list of the new blocks would help:
>>>
>>>     1. Burst/Gap Loss Summary Statistics Metrics Block
>>>     2. Burst/Gap Discard Summary Statistics Metrics Block
>>>     3. Frame Impairment Statistics Summary Metrics Block
>>>
>>> back to the existing text:
>>>    The first two block types support the reporting of burst gap loss/
>>>    discard summary statistics including packet loss/discard proportion,
>>>    mean and variance and belong to the class of transport-related end
>>>    system metrics defined in [RFC6792].  These two blocks are intended
>>>    to be used in conjunction with information from the Burst Gap Loss
>>>    Metrics Block or Burst Gap Discard Metrics Block, and on which these
>>>    two block therefore depend.  
>>> add a reference to [RFC3611] for these two blocks.
>>>
>>>    The metrics in the Burst Gap Loss block
>>>    or Burst Gap Discard Metrics Block can be used independently of the
>>>    metrics defined in the first two blocks.
>>>
>>> That's obvious, because anyone implementing RFC3611 is already doing it.
>>>
>>> The authors appear to have avoided using "video" as an adjective when 
>>> referring to video frames throughout the memo. For example:
>>>
>>>    The third block supports the reporting of detailed statistics for
>>>    each frame type, including the number of frames received, lost and
>>>    discarded of each frame type in the Group of Pictures (GOP) and
>>>    additional data allowing the calculation of statistical parameters
>>>    (e.g.,the proportion of each frame type impaired by packet loss and
>>>    discard).  The metrics defined in this block belong to the class of
>>>    application layer metrics defined in [RFC6792].
>>>
>>> These are all video frames above, no?  At least say so once.
>>>
>>> Section 2.1, on Terminology, says
>>>
>>>     Picture Type
>>>
>>>       Picture Types used in the different video algorithms are composed
>>>       of the Key frame and Derived frames.  The Key frame is also called
>>>       a reference frame and used as a reference for predicting other
>>>       pictures.  It is coded without prediction from other pictures.
>>>       Derived frames are derived from a Key frame using a prediction
>>>       algorithm.
>>>
>>> At first this seems quite generic, but the definition may fail when one slice
>>> of the video frame is independently coded, and other slices are predictively
>>> coded (this has been done to smooth the video rate). Is any video frame that
>>> includes independent or reference coding a Key frame?
>>>
>>> Section 3.1.2 says:
>>>    Burst Loss Rate: 16 bits
>>>
>>>       The fraction of packets lost during bursts since the beginning of
>>>       reception, expressed as a fixed point number with the binary point
>>>       at the left edge of the field.  This value is calculated by
>>>       dividing Packets Loss in Bursts by Total Packets expected in
>>>       Bursts as follows:
>>>           Packets Loss in Bursts / Total Packets expected in Bursts
>>>
>>> How is a burst loss ratio = 1.0 reported?  All the digits are to the 
>>> right of the decimal place.
>>>
>>>    Burst Duration Mean:16bits
>>>
>>>       The mean burst duration is obtained as the quotient:
>>>
>>>       mean = Sum of Burst Durations / Number of Bursts
>>>
>>> How is Divide by Zero handled?
>>>
>>>    Burst Duration Variance:16bits
>>>
>>>       The variance of the burst duration is obtained using the standard
>>>       result:
>>>
>>>       var = ( Sum of Squares of Burst Durations - Number of Bursts *
>>>       mean^2 ) / (Number of Bursts - 1)
>>>
>>> How is Divide by Zero handled? (e.g., Number of Bursts = 1)
>>>
>>> Section 4.1.2 says
>>>
>>>    Number of full frames lost (lost_full_frames): 32 bits
>>>
>>>       If one frame is completely lost, this frame is regarded as one
>>>       lost full frame.  The lost_full_frames is equivalent to the number
>>>       of full frames lost in the above sequence number interval.
>>>
>>> Is this a metric definition, fully lost video frames? 
>>> How is this event detected so it can be counted?
>>> Is there a reference for this metric available?
>>>
>>>    Number of partial frames lost (lost_partial_frames): 32 bits
>>>
>>>       If one frame is partially lost, this frame is regarded as one lost
>>>       fractional frame.  The value of the lost_partial_frames field is
>>>       equivalent to the number of partial frames lost in the above
>>>       sequence number interval.
>>>
>>> Is this a metric definition, partially lost video frames? 
>>> How is this event detected so it can be counted?
>>> Is there a reference for this metric available?
>>>
>>> -=-=-=-=-=-=-
>>>
>>>> -----Original Message-----
>>>> From: [e-mail] [mailto:[e-mail]] On Behalf
>>>> Of Benoit Claise
>>>> Sent: Sunday, January 27, 2013 2:09 PM
>>>> To: [e-mail]
>>>> Cc: [e-mail]; draft-ietf-xrblock-rtcp-xr-summary-
>>>> [e-mail]; Gonzalo Camarillo
>>>> Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
>>>> xrblock-rtcp-xr-summary-stat-06
>>>>
>>>> Dear all,
>>>>
>>>> Here is an official request for the RFC 6390 review.
>>>> Who would like to volunteer?
>>>>
>>>> Regards, Benoit
>>>>> Hi Benoit,
>>>>>
>>>>> as suggested by Dan in his PROTO writeup, I would like to request the
>>>>> performance metrics directorate to perform an RFC 6390 review of the
>>>>> following draft as part of its IETF LC:
>>>>>
>>>>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-
>>>> stat/
>>>>> Could you please arrange such a review?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Gonzalo
>>>>>
>>>>>
>>>> _______________________________________________
>>>> pm-dir mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/pm-dir
>>
>>
>>
>> _______________________________________________
>> pm-dir mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/pm-dir
>