[AVTCORE] FW: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts
"Charles Eckel (eckelcu)" <eckelcu@cisco.com> Thu, 10 November 2011 18:29 UTC
Return-Path: <eckelcu@cisco.com>
X-Original-To: avt@ietfa.amsl.com
Delivered-To: avt@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F420C21F8B70; Thu, 10 Nov 2011 10:29:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.998
X-Spam-Level:
X-Spam-Status: No, score=-3.998 tagged_above=-999 required=5 tests=[AWL=-2.043, BAYES_00=-2.599, FRT_PENIS1=3.592, J_CHICKENPOX_73=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4, SARE_SUB_ENC_UTF8=0.152]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L-utq3REGsUH; Thu, 10 Nov 2011 10:29:28 -0800 (PST)
Received: from mtv-iport-4.cisco.com (mtv-iport-4.cisco.com [173.36.130.15]) by ietfa.amsl.com (Postfix) with ESMTP id 90FE821F8B63; Thu, 10 Nov 2011 10:29:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=eckelcu@cisco.com; l=26590; q=dns/txt; s=iport; t=1320949768; x=1322159368; h=mime-version:content-transfer-encoding:subject:date: message-id:from:to:cc; bh=+GMXAd5qG+XQSORlLQTYvsDsYECa/GTSs/lJSXoq23g=; b=CaFGdBIJmom8v2xGrNIyZv46wuvTFnD92f4Ad9VNG6/AEjVU4Y+zyFai AksIUitszXDvVA2RHa1TLbTuhDyL4T/fHsj9QSgs0J/jR2xOF3A8GBGUF gelkSkKUZ28k83caBL3bHos70ToOGu8fNdv1dtWCYfjvYwTamKOCl3RHS Q=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AsIAANYWvE6rRDoG/2dsb2JhbABABIR9lTWOdYEFgQWBcgEBAQQBAQEPARANBDoLDAYBBgIRBAEBAQICBgYXAQICAwElHwYBAQEFBAEEEwgah2iZPwGMWZIHgTCFKYIPM2MEiA+RWoUBh1Q
X-IronPort-AV: E=Sophos;i="4.69,489,1315180800"; d="scan'208";a="13495755"
Received: from mtv-core-1.cisco.com ([171.68.58.6]) by mtv-iport-4.cisco.com with ESMTP; 10 Nov 2011 18:29:27 +0000
Received: from xbh-sjc-221.amer.cisco.com (xbh-sjc-221.cisco.com [128.107.191.63]) by mtv-core-1.cisco.com (8.14.3/8.14.3) with ESMTP id pAAITRtU029205; Thu, 10 Nov 2011 18:29:27 GMT
Received: from xmb-sjc-234.amer.cisco.com ([128.107.191.111]) by xbh-sjc-221.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 10 Nov 2011 10:29:27 -0800
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
Date: Thu, 10 Nov 2011 10:29:24 -0800
Message-ID: <E1CBF4C7095A3D4CAAAEAD09FBB8E08C05C12E11@xmb-sjc-234.amer.cisco.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts
Thread-Index: AcyfQOikAGDHnzyVRnO4EJ69oXkSeQAlTGXg
From: "Charles Eckel (eckelcu)" <eckelcu@cisco.com>
To: avt@ietf.org
X-OriginalArrivalTime: 10 Nov 2011 18:29:27.0075 (UTC) FILETIME=[AD920F30:01CC9FD6]
Cc: xrblock <xrblock@ietf.org>
Subject: [AVTCORE] FW: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts
X-BeenThere: avt@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/avt>, <mailto:avt-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/avt>
List-Post: <mailto:avt@ietf.org>
List-Help: <mailto:avt-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 Nov 2011 18:29:30 -0000
(as xrblock chair) Forwarding to avtcore list as the comments are in regard to an existing AVTCORE working group draft. I have cc'd xrblock such that folks on xrblock know to post any further comments on this thread to avtcore. Cheers, Charles -----Original Message----- From: Qin Wu [mailto:bill.wu@huawei.com] Sent: Wednesday, November 09, 2011 4:37 PM To: Charles Eckel (eckelcu); Alan Clark; zhaojing@sttri.com.cn Cc: xrblock Subject: Re: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts Yes, that's what I think to add one such definition to draft-ietf-avtcore-monarch. Thank for your reminding. Regards! -Qin ----- Original Message ----- From: "Charles Eckel (eckelcu)" <eckelcu@cisco.com> To: "Qin Wu" <bill.wu@huawei.com>; "Alan Clark" <alan.d.clark@telchemy.com>; <zhaojing@sttri.com.cn> Cc: "xrblock" <xrblock@ietf.org> Sent: Thursday, November 10, 2011 5:56 AM Subject: Re: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts > (As an individual) > > It looks good to me as well. Furthermore, it seems these comments are applicable to monitoring metrics in general, not just those for QoE. That being the case, is the intention to add these definitions into draft-ietf-avtcore-monarch? > > Cheers, > Charles > >> -----Original Message----- >> From: xrblock-bounces@ietf.org [mailto:xrblock-bounces@ietf.org] On Behalf Of Qin Wu >> Sent: Tuesday, November 08, 2011 9:51 PM >> To: Alan Clark; zhaojing@sttri.com.cn >> Cc: xrblock >> Subject: Re: [xrblock]答复: Re: 答复: General comments on xrblock metrics drafts >> >> Your proposal looks good to me. >> Thanks! >> >> Regards! >> -Qin >> >> ----- Original Message ----- >> From: Alan Clark <mailto:alan.d.clark@telchemy.com> >> To: Qin Wu <mailto:bill.wu@huawei.com> ; zhaojing@sttri.com.cn >> Cc: xrblock <mailto:xrblock@ietf.org> >> Sent: Tuesday, November 08, 2011 9:36 PM >> Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts >> >> Hi Qin >> >> This is an alternative to interval or sample and hence the three of them need to be enumerated. >> >> Interval Metric flags (I): 2 bits >> >> This field is used to indicate whether the ???????????? >> metrics block is an Interval, Cumulative or Sampled report, that is, >> whether the reported values apply to the most recent measurement >> interval duration between successive metrics reports (I=10) (the >> Interval Duration) or to the accumulation period characteristic of >> cumulative measurements (I=00) (the Cumulative Duration) or >> to the value of a continuously measured or calculated that has been >> sampled at end of the interval (I=01) (Sampled Value). >> >> Best Regards >> >> Alan >> >> >> On 11/7/11 8:43 PM, "Qin Wu" <bill.wu@huawei.com> wrote: >> >> >> >> Hi, Alan: >> Based on our discussion, would you like to propose the change to the drafts regarding >> sample metric? >> e.g., what's your proposed change to the Interval metric flag? or just allocate additional >> falg to sample? >> Also it will be great you provide the clear definition of sample metric. >> My understanding to sample metrics is: >> " >> Sample metrics >> >> It is referred to the metrics of which the reported values only >> apply to the sample instant which is any time of the measurement duration. >> " >> If this is not precise, please correct me. Thanks! >> >> Regards! >> -Qin >> >> >> >> ----- Original Message ----- >> >> From: Alan Clark <mailto:alan.d.clark@telchemy.com> >> >> To: Qin Wu <mailto:bill.wu@huawei.com> ; zhaojing@sttri.com.cn >> >> Cc: xrblock <mailto:xrblock@ietf.org> >> >> Sent: Friday, November 04, 2011 10:07 AM >> >> Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts >> >> >> Hi Qin >> >> The goal in establishing a framework should be to make provision for things that >> you would reasonably expect. There are quite a few continuously varying parameters that could be >> reported using a sampled approach - audio signal level, noise level, video bandwidth to name a few. >> >> The first part of the header comprises >> - Block type (8 bits) >> - Interval/Cumulative ( 1 bit) >> - Tag (3 bits) >> - Reserved (4 bits) >> >> Expanding the Interval/Cumulative to 2 bits still leaves 3 reserved bits, so this >> would not break the budget. >> >> Best Regards >> >> Alan >> >> >> >> >> On 11/3/11 9:26 PM, "Qin Wu" <bill.wu@huawei.com> wrote: >> >> >> >> >> Hi, Alan: >> I tend to agree with you. But currently I don't see the sampled metric is >> applied to all the metrics except PDV metric, >> So my proposal is to add one bit in PDV metric block header to indicate if >> sample will be used. >> If there is any future metric that need to support sample, we can add >> indication bit later. >> >> Regards! >> -Qin >> >> >> >> >> ----- Original Message ----- >> >> From: Alan Clark <mailto:alan.d.clark@telchemy.com> >> >> To: Qin Wu <mailto:bill.wu@huawei.com> ; zhaojing@sttri.com.cn >> >> Cc: xrblock <mailto:xrblock@ietf.org> >> >> Sent: Thursday, November 03, 2011 8:26 PM >> >> Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock >> metrics drafts >> >> >> Qin >> >> You cannot derive the value of a sampled metric from two interval >> metrics. >> >> A sampled metric is useful for measuring and reporting on parameters >> that vary in time. Sometimes it is acceptable to report the min/max/average of a time varying >> parameter and sometimes you may want to report the value at points in time. >> >> At risk of causing complete confusion - I’m going to use a temperature >> example. >> >> (i) Between 2pm and 3pm the average temperature was 10 degrees C, is >> an interval metric >> (ii) Between 2pm and 3pm the maximum temperature was 11 degrees C, is >> an interval metric >> (ii) The temperature at 3pm is 2 degrees C is a sampled measurement >> >> It is remarkably easy to add one bit to the XRBlock head at this time >> to allow for current or future metrics that are sampled. Given that there are existing RTP examples >> of this type of metric then we know that it is already useful to report sampled values. >> >> Best Regards >> >> Alan >> >> On 11/3/11 3:15 AM, "Qin Wu" <bill.wu@huawei.com> wrote: >> >> >> >> >> >> Hi, Alan: >> Thank for your clarification on the difference between Interval >> metric, Cumulative metric the existing draft described and sampled metric you proposed. >> It looks Sample metric is established upon the Interval metric >> and present different features of any transport impairments from Culmulative metric. >> e.g., we have a list of Interval metric >> x1, x2, x3,x4,x5,x6,x7,x8,x9,x10 >> The sample metric is as follows: >> S1,S2,S3,S4,S5,S6,S7,S8,S9,S10 >> So the sampled metic can be calculated using the following >> example: >> S1=x1; S2= func(x1,x2); S3=func(x2,x3); S4=func(x3,x4);... >> In this example, Si only rely on the xi and xi-1. >> >> So the question is: >> Does sample metric fix the problem you raised below for >> Cumulative metric? >> >> Also what's the sampled algorithm used for calculating such >> metric and presented at the end of each Interval? the algorithm similar to PPDV? or any algorithm >> we can find? >> Do you think measurement Interval for Interval metric is same as >> one for Sampled metric? >> >> It looks to me that it is not that simple to just add one more >> bit in the XRBlock header to indicate Sampled metric is used? You should make sure the measurement >> interval and specific sampled algorithm? >> But Interval metric and Cumulative metric don't have such >> issue. >> >> >> Regards! >> -Qin >> >> >> >> >> >> ----- Original Message ----- >> >> From: Alan Clark <mailto:alan.d.clark@telchemy.com> >> >> To: Qin Wu <mailto:bill.wu@huawei.com> ; >> zhaojing@sttri.com.cn >> >> Cc: xrblock <mailto:xrblock@ietf.org> >> >> Sent: Thursday, November 03, 2011 10:41 AM >> >> Subject: Re: [xrblock] 答复: Re: 答复: General comments on >> xrblock metrics drafts >> >> >> Qin >> >> An interval metric is measured over a time interval. A >> cumulative metric is the same as an interval metric if there is only one interval. >> >> A metric for an interval should not be affected by >> previous intervals. >> >> A “sampled” metric is the value of a metric at the >> instant of sampling. >> >> (a) Simple case >> >> If we consider 10 intervals, each with 1000 packets >> expected. Intervals 1 and 3 have 100 packets missing. >> >> Cumulative packet loss rate, reported at the end of each >> interval would be >> >> 10%, 5%, 6.7%, 5%, 4%, 3.3%, 2.9%, 2.5%, 2.2%, 2.0% >> >> Interval packet loss rate would be >> 10%, 0%, 10%, 0%, 0%, 0%, 0%, 0%, 0%, 0% >> >> Averaging the interval loss rates gives (10+10)/10 = >> 2.0% >> >> calculating the running average loss rate and reporting >> sampled values would give something roughly like >> >> 10%, 3%, 10%, 3%, 1%, 0.1%, 0.01%, 0%, 0%, 0% >> >> (b) Slightly less simple case >> >> Assume the same intervals and loss rates however say that >> some of the packets from interval 1 were not actually lost, but arrive in interval 2. So at the >> time the packet loss rate was calculated for interval 1 it appeared to be 8% instead of 10%. >> >> We don’t typically report negative loss rates, so the >> interval loss rate would be reported as >> >> 8%, 0%..... >> >> If we compute loss rate cumulatively then this problem >> would only occur at the end of the cumulative period, so the loss rates reported would be >> >> 8%, 5%, ... >> >> In other words the cumulative metric corrects itself. >> >> This would mean that if we added the interval packet loss >> rates together the sum may not match the cumulative value. >> >> Best Regards >> >> Alan >> >> >> On 11/2/11 3:11 AM, "Qin Wu" <bill.wu@huawei.com> wrote: >> >> >> >> >> >> >> I tend to agree with you. But It looks to me >> Interval metric also can be used to calculate cumulative charastertistics. The only difference to >> the Cumulative metric is one has short interval, one has longer interval. >> >> Am I right? >> >> >> >> Or should we view Interval metric as a metric that >> is used to report instant value at the end of the interval set for the interval metric? in that >> case, there will be no difference between Iinteval metric and Sampled metric, >> >> >> >> Or should we view Interval metric as a basic metric >> that can be directly measured and calculated and does not rely on other metric. >> >> which one is correct? >> >> >> >> Before answering this question, I think >> introducing new term "sampled metric" is very confused. >> >> >> >> Regards! >> >> -Qin >> >> >> >> >> >> ________________________________ >> >> 发件人: zhaojing@sttri.com.cn [zhaojing@sttri.com.cn] >> 发送时间: 2011年11月2日 14:59 >> 到: Alan Clark; Qin Wu >> Cc: xrblock >> 主题: Re: Re: 答复: [xrblock] General comments on >> xrblock metrics drafts >> >> I know you intend to seperate Sampled metric from >> Interval Metric. However according to the defintion of Interval metric definition: >> " >> The Interval metric is refered to as the reported >> values apply to the most recent measurement >> interval duration between successive metrics >> reports >> " >> In your understanding, the samepled metric is >> referred to as the measured value is sampled at the end of interval or apply to the end of >> interval. >> Is that what you say? >> But I am not convinced. It looks to me the sampled >> metric can be classified into cumulative metric. >> Take PPDV as example, the J(i)= J(i-1)++ (|D(i- >> 1,i)| - J(i-1))/16 >> Isn't J(i) a cumulative metric since the J(i) is the >> accumulation period characteristic that combine J(i-1),J(i-2),..J(0) >> during cumulative duration? >> Cheers! >> Jing >> ------------------ 原始邮件 ------------------ >> >> 发件人：Alan Clark >> 时 间：2011/11/02 05:25:01 星期三 >> 收件人：zhaojing@sttri.com.cn, Qin Wu >> 抄送人：xrblock >> 主 题：Re: 答复: [xrblock] General comments on >> xrblock metrics drafts >> >> Jing >> >> An Interval metric is calculated over the interval >> - so packet loss for an interval would be the packets lost during the interval divided by the >> number of packets expected. >> >> If the packet loss rate was continuously calculated >> as a running average (updated per packet) and the value of this packet loss rate metric was >> sampled at the end of the interval, it would be a sampled value of the metric and not the >> interval value. >> >> Hence there is a difference between an interval >> metric and a sampled metric, which IMHO means that we should have a way to state that a metric is >> Cumulative, Interval or Sampled. >> >> Best Regards >> >> Alan >> >> >> On 11/1/11 4:21 AM, "zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> " >> <zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> > wrote: >> >> >> >> >> >> >> >> >> >> Are you saying the sample semantics has already been incorporate into Interval/Cumulative >> indication? >> >> >> >> Cheers! >> >> Jing >> >> ------------------ 原始邮件 ------------------ >> >> 发件人：Qin Wu >> 时 间：2011/11/01 15:19:54 星期二 >> 收件人：Alan Clark , zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> >> 抄送人：xrblock >> 主 题：答复: [xrblock] General comments on xrblock metrics drafts >> >> >> >> >> >> >> Hi, Alan: >> >> >> So the Interval metric you understand is for per RTP packet monitoring or observation >> while the sampled metric is not. >> In other words, >> a. The Interval metric is obtained by keeping trace of each RTP packet during a specified >> Interval and treat all the sequence number valid. >> b.The sampled metric is obtained by sampling a sub set of RTP packets during a specified >> Interval. >> Is that what you say? >> >> >> >> >> >> However according to PDV drafts and other similar xrblock metrics drafts, per-packet >> statistics doesn't apply to these metrics(See security consideration section). >> >> >> That seems to implicate that both Interval metrics and cumulative metric are not used for >> per-packet accounting but for sampling use. >> >> >> >> >> >> Regards! >> >> >> -Qin >> >> >> >> >> >> >> ________________________________ >> >> >> 发件人: xrblock-bounces@ietf.org >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/xrblock-bounces@ietf.org> >> [xrblock-bounces@ietf.org >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/xrblock-bounces@ietf.org> ] 代表 >> Alan Clark [alan.d.clark@telchemy.com >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/alan.d.clark@telchemy.com> ] >> 发送时间: 2011年10月31日 20:08 >> 到: zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> >> Cc: xrblock >> 主题: Re: [xrblock] General comments on xrblock metrics drafts >> >> >> >> >> >> Hi Jing >> >> An interval metric should be measured throughout an interval however a sampled metric is an >> instantaneous value. >> >> Using Jitter/PDV as an example: >> >> - Interval PDV should be calculated by measuring the delay variation of each packet and >> averaging these over the interval. This requires the implementation to sum the individual >> measurements and divide by the number of measurements. >> >> - PPDV (RFC3550) is calculated by keeping a running average of the individual delay >> variation measurements using a scaling factor of 16 and then reporting the value of this running >> average at the end of the interval. With a scaling factor of 16 the PPDV metric is most strongly >> affected by the delay variation of the most recent 10-20 packets and hence is not representative >> of the whole interval. >> >> Interval averages can be quite misleading if the measurements are strongly time varying, >> which PDV typically is. >> >> Best Regards >> >> Alan >> >> >> On 10/31/11 5:23 AM, "zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> >> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <zhaojing@sttri.com.cn >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@sttri.com.cn> >> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote: >> >> >> >> >> >> >> Hi, Alan: >> >> Question for clarification. >> >> Isn't the sampled metric same as the Interval metric? Is the sampled metric only applied to >> per-packet reporting ? what's the difference between sampled metric and non-sampled metric? >> >> what's the difference between sampled metric and Interval metric? Would you like to >> clarify more? >> >> >> >> Cheers! >> >> Jing >> >> ------------------ 原始邮件 ------------------ >> >> 发件人：Alan Clark >> 时 间：2011/10/31 06:51:03 星期一 >> 收件人：xrblock@ietf.org >> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件人：xrblock@ietf.org> >> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> >> 抄送人： >> 主 题：[xrblock] General comments on xrblock metrics drafts >> >> >> 1. Cumulative, Interval and Sample metrics >> The drafts current have one bit assigned for Cumulative/Interval indication.... Some >> metrics are however sampled and don’t have validity for an interval or a call/session; an example >> of this is PPDV, which is a sampled value of a running average and only has validity for the last >> 200-300ms before the value is sampled. >> Proposal - increase this to two bits to allow for the indication of a Sampled value. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ________________________________ >> >> >> >> _______________________________________________ >> xrblock mailing list >> xrblock@ietf.org >> https://www.ietf.org/mailman/listinfo/xrblock >> >> >> >> >> >> >> >> > > _______________________________________________ > xrblock mailing list > xrblock@ietf.org > https://www.ietf.org/mailman/listinfo/xrblock >
- [AVTCORE] FW: [xrblock]答复: Re: 答复: General commen… Charles Eckel (eckelcu)