Re: [xrblock] WGLC for draft-ietf-xrblock-rtcp-xr-discardandxrblock-rtcp-xr-discard-rle-metrics

[Alan Clark <[e-mail]>]

The key is that from a performance perspective you would need to look at
early/ late discards as a symptom of PDV due to congestion (or route
changes) however duplicate packets have quite different causes.

(a) A few duplicate packets can indicate some form of Layer 1/2 LAN problem.
This would not need to be an accurate measure - more of a general barometer.

(b) If the number of duplicate packets is very high then this may be due to
RTP replication - and if this is the case then you would want to compare the
number of duplicate packets to the number of received packets in the same
time interval. If the duplicate packet count is X% of the received packet
count this indicates that a (100-X)% packet loss rate is being "hidden" by
the replicated packets, and it is very useful to know the actual loss rate
(useful to indicate that replication should be kept "on" and helpful to know
that there are some network issues that need to be investigated).

It may be useful to explain this in the draft to provide some background

Regards

Alan 


On 7/9/12 8:59 PM, "Shida Schubert" <[e-mail]> wrote:

> 
> Hi Qin;
> 
>  I also think (a) is more useful. (b) seems to merge
> 4 different semantics into 1 (discard + early, discard  + late,
> discard + both, discard only).
> 
>  I think all we need to do, is to add some text describing
> that misc and other 3 can not be in a same reporting
> block, but you can convey them by using 2 reporting
> blocks instead for the same reporting period.
> 
>  Regards
>   Shida
> 
> On Jul 9, 2012, at 7:39 PM, Varun Singh wrote:
> 
>> Hi Qin,
>> 
>> I thought the agreement for some time has been that the "others"
>> category was not a summation of everything but an independent category
>> for discards. That was one of the reasons why I had originally
>> proposed the order of discard types to be misc (DT=0) and then early
>> (DT=01), late (DT=10) and both (DT=11). However, I have no strong
>> opinion in the ordering.
>> 
>> I prefer the proposal (a) because there is the duplicate RTP streams
>> draft (http://tools.ietf.org/html/draft-ietf-avtext-rtp-duplication)
>> and while I have not implemented it, but an RTP monitors might want to
>> know which streams duplicate packets were discarded independently of
>> late/early arrivals.
>> 
>> Cheers,
>> Varun
>> 
>> On Mon, Jul 9, 2012 at 10:59 AM, Qin Wu <[e-mail]> wrote:
>>> Hi, Shida:
>>> Just to clarify, the question you ask is very good question.
>>> That's why Varun and I both proposed some text on the list try to fix the
>>> issue you raised.
>>> Currently, one thing I am not sure is whether we should report
>>> (a) discards of duplication packets independently (As Alan suggested)
>>> or
>>> (b) report discards of duplication packets combined with early and discard
>>> (i.e.,As I proposed in the current draft, DT=3 for all discard types).
>>> 
>>> or we take both (a) and (b) in the draft.
>>> 
>>> Regards!
>>> -Qin
>>> ----- Original Message -----
>>> From: "Shida Schubert" <[e-mail]>
>>> To: "Qin Wu" <[e-mail]>
>>> Cc: "Varun Singh" <[e-mail]>; "Alan Clark"
>>> <[e-mail]>; "xrblock" <[e-mail]>
>>> Sent: Monday, July 09, 2012 3:31 PM
>>> Subject: Re: [xrblock] WGLC for
>>> draft-ietf-xrblock-rtcp-xr-discardandxrblock-rtcp-xr-discard-rle-metrics
>>> 
>>> 
>>> 
>>> Qin;
>>> 
>>> I was just simply asking a question that came up while I was reviewing the
>>> draft
>>> and I have no position on this.
>>> 
>>> So what you have currently is fine with me.
>>> 
>>> Regards
>>>  Shida
>>> 
>>> On Jul 9, 2012, at 2:59 PM, Qin Wu wrote:
>>> 
>>>> Hi,Shida:
>>>> Yes, currently we have four types. but the the 4 th type has been replaced
>>>> with combine early, late and discarded due to duplication) in the current
>>>> draft.
>>>> You are right, if we combine discards due to duplication with either 1, or
>>>> 2, we need to have two report blocks.
>>>> My question, do we have the clear use case for such combinations you
>>>> identify?
>>>> 
>>>> Regarding the assumption on whether it is used when duplicate packet
>>>> arrives early or late,
>>>> I think you should also consider one duplicated packet arrives on  time and
>>>> how is discareded due to that it is duplicated packet.
>>>> 
>>>> Regards!
>>>> -Qin
>>>> ----- Original Message -----
>>>> From: "Shida Schubert" <[e-mail]>
>>>> To: "Varun Singh" <[e-mail]>
>>>> Cc: "Alan Clark" <[e-mail]>; "Qin Wu"
>>>> <[e-mail]>; "xrblock" <[e-mail]>
>>>> Sent: Saturday, July 07, 2012 8:13 AM
>>>> Subject: Re: [xrblock] WGLC for
>>>> draft-ietf-xrblock-rtcp-xr-discardandxrblock-rtcp-xr-discard-rle-metrics
>>>> 
>>>> 
>>>> 
>>>> So my question was..
>>>> 
>>>> From what I can read, currently we have four types.
>>>> 
>>>> 1. early ony
>>>> 2. late only
>>>> 3. both (late / early)
>>>> 4. other (discarded to duplicate etc.)
>>>> 
>>>> According to my understanding based on Qin's response.
>>>> 
>>>> When there is an occurrence of 4 along with 1,2 or 3, you
>>>> need to have 2 report blocks.
>>>> 
>>>> 1 and 4, 2 and 4 or 3 and 4.. since we don't have a way
>>>> to express these combination currently..
>>>> 
>>>> This is based on my assumption that other is distinguished
>>>> from early or late Or is it used when duplicate packet arrives
>>>> early or late?
>>>> 
>>>> Regards
>>>> Shida
>>>> 
>>>> On Jul 6, 2012, at 8:34 PM, Varun Singh wrote:
>>>> 
>>>>> On Fri, Jul 6, 2012 at 1:51 PM, Alan Clark <[e-mail]>
>>>>> wrote:
>>>>>> Shida
>>>>>> 
>>>>>> You could have all three types of discard occurring within a single
>>>>>> stream.
>>>>>> For example - if RTP replication is used for resilience then every
>>>>>> interval
>>>>>> would have the same number of duplicate/other packets as data packets and
>>>>>> if
>>>>>> there is a high level of jitter then there would also be late packets,
>>>>>> early
>>>>>> packets or both.
>>>>>> 
>>>>> 
>>>>> I agree that one of early, late or others or any combination of the
>>>>> three is a valid reporting case.
>>>>> Perhaps there needs to be only a clarification on when to use early
>>>>> and late or combined early and late.
>>>>> 
>>>>> Regards,
>>>>> Varun
>>>>> 
>>>>>> Best Regards
>>>>>> 
>>>>>> Alan
>>>>>> 
>>>>>> 
>>>>>> On 7/6/12 5:07 AM, "Shida Schubert" <[e-mail]> wrote:
>>>>>> 
>>>>>>> 
>>>>>>> (as contributor)
>>>>>>> 
>>>>>>> So does that mean that in a single report block, early and late can
>>>>>>> co-exist
>>>>>>> when it is described as type "both" but you can't have "other" + early,
>>>>>>> "other" + late or "other" + both?
>>>>>>> 
>>>>>>> Thus, for the same reporting period, you would have separate reporting
>>>>>>> block for the above discarded packets combination? If that is the case,
>>>>>>> I think this should be explicitly stated in the section where the type
>>>>>>> is
>>>>>>> described.
>>>>>>> 
>>>>>>> Also, the draft reads like it is dependent on Measurement Identity based
>>>>>>> on the sub-section "number of packets discarded". If that is the case,
>>>>>>> the
>>>>>>> Mesurement Identity should become a Normative Reference.
>>>>>>> 
>>>>>>> Regards
>>>>>>> Shida
>>>>>>> 
>>>>>>> On Jul 3, 2012, at 11:09 AM, Qin Wu wrote:
>>>>>>> 
>>>>>>>> That's what I think, thank for your clarification.
>>>>>>>> 
>>>>>>>> Regards!
>>>>>>>> -Qin
>>>>>>>> ----- Original Message -----
>>>>>>>> From: "Alan Clark" <[e-mail]>
>>>>>>>> To: "Dan (Dan)" <[e-mail]>; "Qin Wu" <[e-mail]>;
>>>>>>>> "Shida
>>>>>>>> Schubert" <[e-mail]>; "xrblock" <[e-mail]>
>>>>>>>> Cc: <[e-mail]>
>>>>>>>> Sent: Monday, July 02, 2012 7:49 PM
>>>>>>>> Subject: Re: [xrblock] WGLC for
>>>>>>>> 
draft-ietf-xrblock-rtcp-xr-discardandxrblock-rtcp-xr-discard-rle-metric>>>>>>>>
s
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Hi Dan
>>>>>>>>> 
>>>>>>>>> There are some implementations of RTP that send duplicate packets (in
>>>>>>>>> some
>>>>>>>>> cases every packet) in order to provide a simple form of FEC.
>>>>>>>>> Reporting
>>>>>>>>> duplicate packets as "duplicates" can allow the user to determine what
>>>>>>>>> proportion of lost packets are being concealed by the process.  For
>>>>>>>>> example,
>>>>>>>>> if I send 1000 packets but duplicate these in order to provide FEC -
>>>>>>>>> then
>>>>>>>>> knowing that 900 duplicate packets were discarded tells me that the
>>>>>>>>> network
>>>>>>>>> packet loss rate was 10%.
>>>>>>>>> 
>>>>>>>>> The reason that RFC3611 excluded duplicates was that the discard count
>>>>>>>>> was
>>>>>>>>> intended to show what effect late/early arriving packets were having
>>>>>>>>> on the
>>>>>>>>> quality perceived by the user.  Discarded duplicates have no effect
>>>>>>>>> whereas
>>>>>>>>> a discarded late packet causes a "hole" in the decoded stream that has
>>>>>>>>> to be
>>>>>>>>> repaired by PLC
>>>>>>>>> 
>>>>>>>>> It is useful to report discards of duplicate packets "separately from"
>>>>>>>>> the
>>>>>>>>> early/late arrival discard count. They should not be combined into the
>>>>>>>>> same
>>>>>>>>> counter.  This means that the early/late arrival discard count would
>>>>>>>>> be
>>>>>>>>> consistent with RFC3611 but there is an additional count of discarded
>>>>>>>>> duplicate packets
>>>>>>>>> 
>>>>>>>>> Best Regards
>>>>>>>>> 
>>>>>>>>> Alan
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 7/2/12 6:52 AM, "Dan (Dan)" <[e-mail]> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi Qin,
>>>>>>>>>> 
>>>>>>>>>> Thank you for your response.
>>>>>>>>>> 
>>>>>>>>>> I am fine with your proposed resolutions with the exception of item
>>>>>>>>>> 3.
>>>>>>>>>> The resolution proposed by you suggests including packets 'thrown
>>>>>>>>>> away
>>>>>>>>>> before playout (e.g., packet duplication or redundancy)' in the
>>>>>>>>>> discard
>>>>>>>>>> count metric. This would make the discard count metric inconsistent
>>>>>>>>>> to
>>>>>>>>>> the discard rate metric defined in section 4.7.1 of RFC 3611 which
>>>>>>>>>> explicitly excludes duplicate packet discards.
>>>>>>>>>> 
>>>>>>>>>> Am I the only one (exaggeratedly) concerned by this inconsistency? I
>>>>>>>>>> would love to hear other opinions.
>>>>>>>>>> 
>>>>>>>>>> Dan
>>>>>>>>>> (speaking as contributor)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Qin Wu [mailto:[e-mail]]
>>>>>>>>>>> Sent: Friday, June 29, 2012 6:33 AM
>>>>>>>>>>> To: Romascanu, Dan (Dan); Shida Schubert; xrblock
>>>>>>>>>>> Cc: [e-mail]
>>>>>>>>>>> Subject: Re: [xrblock] WGLC for draft-ietf-xrblock-rtcp-xr-
>>>>>>>>>>> discardandxrblock-rtcp-xr-discard-rle-metrics
>>>>>>>>>>> 
>>>>>>>>>>> Hi,Dan:
>>>>>>>>>>> Thank for your valuable review to
>>>>>>>>>>> draft-ietf-xrblock-rtcp-xr-discard.
>>>>>>>>>>> Please see my replies inline.
>>>>>>>>>>> 
>>>>>>>>>>> Regards!
>>>>>>>>>>> -Qin
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>> From: "Romascanu, Dan (Dan)" <[e-mail]>
>>>>>>>>>>> To: "Shida Schubert" <[e-mail]>; "xrblock"
>>>>>>>>>>> <[e-mail]>
>>>>>>>>>>> Cc: <[e-mail]>;
>>>>>>>>>> <draft-ietf-xrblock-
>>>>>>>>>>> [e-mail]>
>>>>>>>>>>> Sent: Thursday, June 28, 2012 8:02 PM
>>>>>>>>>>> Subject: Re: [xrblock] WGLC for draft-ietf-xrblock-rtcp-xr-
>>>>>>>>>>> discardandxrblock-rtcp-xr-discard-rle-metrics
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> (as contributor)
>>>>>>>>>>>> 
>>>>>>>>>>>> I read the documents and they look almost ready for submission to
>>>>>>>>>> the
>>>>>>>>>>>> IESG.
>>>>>>>>>>>> 
>>>>>>>>>>>> Here are a few comments on draft-ietf-xrblock-rtcp-xr-discard:
>>>>>>>>>>>> 
>>>>>>>>>>>> 1. It would be useful I think to say more about the relation
>>>>>>>>>>>> between
>>>>>>>>>>>> this metric and the discard rate metric defined in section 4.7.1 of
>>>>>>>>>>> RFC
>>>>>>>>>>>> 3611. Maybe calling the metric here Discarded Packets metric would
>>>>>>>>>>> help,
>>>>>>>>>>>> as both RFC 3611 and this document refer to 'discard metric' but
>>>>>>>>>>>> the
>>>>>>>>>>> two
>>>>>>>>>>>> are different (one is rate, the other packets).
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: Good point, I propose to change 'discard metric' in this
>>>>>>>>>> document
>>>>>>>>>>> into 'discard count  metric' since
>>>>>>>>>>> abstract in this draft also uses 'discard count metric'.
>>>>>>>>>>> 
>>>>>>>>>>> To make this consistent with SDP parameter defined in this document,
>>>>>>>>>>> I
>>>>>>>>>>> also like to propose to do the following change
>>>>>>>>>>> OLD TEXT:
>>>>>>>>>>> "
>>>>>>>>>>> xr-format =/ xr-pd-block
>>>>>>>>>>> 
>>>>>>>>>>> xr-pd-block = "pkt-dscrd"
>>>>>>>>>>> 
>>>>>>>>>>> "
>>>>>>>>>>> NEW TEXT:
>>>>>>>>>>> "
>>>>>>>>>>> xr-format =/ xr-pdc-block
>>>>>>>>>>> 
>>>>>>>>>>> xr-pdc-block = "pkt-dscrd-count"
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> "
>>>>>>>>>>> 
>>>>>>>>>>>> 2. In Section 3.1 diagram we use NBGD for Block Type, while the
>>>>>>>>>>>> text
>>>>>>>>>>> in
>>>>>>>>>>>> Section 3.2 refers to the ND constant. We should get to a
>>>>>>>>>>>> consistent
>>>>>>>>>>>> representation
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: It is a typo and will fix this by changing NBGD into ND.
>>>>>>>>>>>> 
>>>>>>>>>>>> 3.
>>>>>>>>>>>> 
>>>>>>>>>>>> Section 2.1:
>>>>>>>>>>>> 
>>>>>>>>>>>>  A packet that arrives within
>>>>>>>>>>>>  this time window but is too early or late to be played out
>>>>>>>>>> shall
>>>>>>>>>>>>  be regarded as discarded.  A packet shall be classified as one
>>>>>>>>>> of
>>>>>>>>>>>>  received (or OK), discarded or lost.  The Discard Metric counts
>>>>>>>>>>>>  only discarded packets.
>>>>>>>>>>>> 
>>>>>>>>>>>> Section 3.1 however includes:
>>>>>>>>>>>> 
>>>>>>>>>>>>     00: packets are discarded due to other reasons than late
>>>>>>>>>>>>     arrival, early arrival, or both (e.g., duplicate, redundant
>>>>>>>>>>>>     packets).
>>>>>>>>>>>> 
>>>>>>>>>>>> This seems inconsistent.
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: Good question. To make them consistent, I propose do the
>>>>>>>>>>> following change to Section 2.1
>>>>>>>>>>> OLD TEXT:
>>>>>>>>>>> "
>>>>>>>>>>>  A packet that arrives within
>>>>>>>>>>>   this time window but is too early or late to be played out shall
>>>>>>>>>>>   be regarded as discarded.  A packet shall be classified as one
>>>>>>>>>> of
>>>>>>>>>>>   received (or OK), discarded or lost.  The Discard Metric counts
>>>>>>>>>>>  only discarded packets.
>>>>>>>>>>> 
>>>>>>>>>>> "
>>>>>>>>>>> NEW TEXT:
>>>>>>>>>>> "
>>>>>>>>>>> A packet that arrives within
>>>>>>>>>>> 
>>>>>>>>>>> this time window but is too early or late to be played out
>>>>>>>>>>> 
>>>>>>>>>>> or is thrown away before playout (e.g., packet duplication or
>>>>>>>>>>> redundancy)
>>>>>>>>>>> 
>>>>>>>>>>> shall be regarded as discarded.  A packet shall be classified as one
>>>>>>>>>> of
>>>>>>>>>>> 
>>>>>>>>>>> received (or OK), discarded or lost.  The Discard Count Metric
>>>>>>>>>>> counts
>>>>>>>>>>> 
>>>>>>>>>>> only discarded packets.
>>>>>>>>>>> "
>>>>>>>>>>> 
>>>>>>>>>>>> 4. Is there any reasons for the Interval Metric flag (I) to be 2
>>>>>>>>>> bits,
>>>>>>>>>>>> rather than one bit, with the other one reserved?
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: Good question, I remembered we got a suggestion on the list
>>>>>>>>>>> before from Kevin Gross which suggested to
>>>>>>>>>>> remove Sampled metric related description from the definition of
>>>>>>>>>>> Interval Metric flag. Since Sampled metric is
>>>>>>>>>>> measured only at a particular time instant however metrics defined
>>>>>>>>>>> in
>>>>>>>>>>> this document is
>>>>>>>>>>> measured over one or several reporting intervals.To get in line with
>>>>>>>>>> the
>>>>>>>>>>> defintion
>>>>>>>>>>> of the Interval Metric flag in other XR BLOCK drafts and address
>>>>>>>>>>> your
>>>>>>>>>>> comment,
>>>>>>>>>>> I propose the following change to the defintion of the interval
>>>>>>>>>>> metric
>>>>>>>>>>> flag:
>>>>>>>>>>> 
>>>>>>>>>>> OLD TEXT:
>>>>>>>>>>> "
>>>>>>>>>>> Interval Metric flag (I): 2 bits
>>>>>>>>>>> 
>>>>>>>>>>>   This field is used to indicate whether the Discard metric is an
>>>>>>>>>>>   Interval or Cumulative metric, that is, whether the reported
>>>>>>>>>>>   values applies to the most recent measurement interval duration
>>>>>>>>>>>   between successive metrics reports (I=10) (the Interval
>>>>>>>>>> Duration)
>>>>>>>>>>>   or to the accumulation period characteristic of cumulative
>>>>>>>>>>>   measurements (I=11) (the Cumulative Duration).
>>>>>>>>>>> 
>>>>>>>>>>> "
>>>>>>>>>>> NEW TEXT:
>>>>>>>>>>> "
>>>>>>>>>>> Interval Metric flag (I): 2 bits
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>   This field is used to indicate whether the Discard Count Metric
>>>>>>>>>> is
>>>>>>>>>>> an
>>>>>>>>>>> 
>>>>>>>>>>>   Interval or Cumulative metric, Sample metric,that is, whether
>>>>>>>>>> the
>>>>>>>>>>> reported
>>>>>>>>>>> 
>>>>>>>>>>>   values applies to the most recent measurement interval duration
>>>>>>>>>>> 
>>>>>>>>>>>   between successive metrics reports (I=10) (the Interval
>>>>>>>>>> Duration)
>>>>>>>>>>> 
>>>>>>>>>>>   or to the accumulation period characteristic of cumulative
>>>>>>>>>>> 
>>>>>>>>>>>   measurements (I=11) (the Cumulative Duration) or is a
>>>>>>>>>>> 
>>>>>>>>>>>   sampled instantaneous value (I=01) (Sampled Value). In this
>>>>>>>>>>> document,
>>>>>>>>>>> 
>>>>>>>>>>>   Discard Count Metric is not measured at a particular time
>>>>>>>>>> instant
>>>>>>>>>>> but over
>>>>>>>>>>> 
>>>>>>>>>>>   one or several reporting intervals. Therefore Discard Count
>>>>>>>>>> Metric
>>>>>>>>>>> MUST not
>>>>>>>>>>> 
>>>>>>>>>>>   be chosen as Sampled Metric.
>>>>>>>>>>> 
>>>>>>>>>>> "
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> 5. number of packets discarded:
>>>>>>>>>>>> 
>>>>>>>>>>>> If the measured value exceeds 0xFFFFFFFD, the value 0xFFFFFFFE
>>>>>>>>>>>>  SHOULD be reported to indicate an over-range measurement.
>>>>>>>>>>> 
>>>>>>>>>>>> Why is this a SHOULD and not a MUST? Are there any exceptions?
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: No,  I will use MUST based on your comment.
>>>>>>>>>>> 
>>>>>>>>>>>> 6. In the IANA Considerations section:
>>>>>>>>>>>> 
>>>>>>>>>>>> s/ The contact information for the registrations is/ The following
>>>>>>>>>>>> contact information is provided for all registrations in this
>>>>>>>>>>> document/
>>>>>>>>>>> 
>>>>>>>>>>> [Qin]: Okay.
>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> xrblock mailing list
>>>>>>>>>> [e-mail]
>>>>>>>>>> https://www.ietf.org/mailman/listinfo/xrblock
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> xrblock mailing list
>>>>>>>>> [e-mail]
>>>>>>>>> https://www.ietf.org/mailman/listinfo/xrblock
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> xrblock mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/xrblock
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> http://www.netlab.tkk.fi/~varun/
>> 
>> 
>> 
>> -- 
>> http://www.netlab.tkk.fi/~varun/
>