Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]
"Weiqiang Sun" <sunwq@MIT.EDU> Thu, 27 August 2009 06:26 UTC
Return-Path: <sunwq@MIT.EDU>
X-Original-To: ccamp@core3.amsl.com
Delivered-To: ccamp@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 10E7928C130; Wed, 26 Aug 2009 23:26:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.598
X-Spam-Level:
X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, STOX_REPLY_TYPE=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hlHWMtSMHljB; Wed, 26 Aug 2009 23:26:12 -0700 (PDT)
Received: from biscayne-one-station.mit.edu (BISCAYNE-ONE-STATION.MIT.EDU [18.7.7.80]) by core3.amsl.com (Postfix) with ESMTP id C4FEA28C114; Wed, 26 Aug 2009 23:26:12 -0700 (PDT)
Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by biscayne-one-station.mit.edu (8.13.6/8.9.2) with ESMTP id n7R6QH83023861; Thu, 27 Aug 2009 02:26:17 -0400 (EDT)
Received: from APC ([202.120.39.240]) (authenticated bits=0) (User authenticated as sunwq@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id n7R6QAoR015983 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 27 Aug 2009 02:26:13 -0400 (EDT)
Message-ID: <1B6858BC083442DE896FA785BC5BD5AE@mit.edu>
From: Weiqiang Sun <sunwq@MIT.EDU>
To: Rschrage <rschrage@schrageconsult.net>, 'Lou Berger' <lberger@labn.net>, 'Henk Uijterwaal' <henk@ripe.net>
References: <FFCC0DAA6C5147538E685E4399AF3272@mit.edu> <000c01ca20d5$03aeca30$0b0c5e90$@net> <EE0622D3476A43FE9DEB4AD90EA91D49@mit.edu> <002601ca265e$89ef67b0$9dce3710$@net>
In-Reply-To: <002601ca265e$89ef67b0$9dce3710$@net>
Date: Thu, 27 Aug 2009 14:26:09 +0800
MIME-Version: 1.0
Content-Type: text/plain; format="flowed"; charset="iso-8859-1"; reply-type="original"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
Importance: Normal
X-Mailer: Microsoft Windows Live Mail 14.0.8064.206
X-MimeOLE: Produced By Microsoft MimeOLE V14.0.8064.206
X-Scanned-By: MIMEDefang 2.42
Cc: ccamp@ietf.org, zhangguoying@mail.ritt.com.cn, 'IETF IPPM WG' <ippm@ietf.org>
Subject: Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]
X-BeenThere: ccamp@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Discussion list for the CCAMP working group <ccamp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ccamp>, <mailto:ccamp-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ccamp>
List-Post: <mailto:ccamp@ietf.org>
List-Help: <mailto:ccamp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ccamp>, <mailto:ccamp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Aug 2009 06:26:13 -0000
Hi Reinhard, Thanks for responding. Please see inline. > > Thanks for sending this part. See below for my responses. > > [RS7] > Is there any help or suggestion on how to actually compute a usable > Lambda_m? > [sunwq] > No. Depending on the performance of the implementation, the lambda_m can > be > very different, even under the same topology. A proper value of Lambda_m > can > only be tuned in the testing. > > [RS7a] > I am a bit at loss here. > > It was my understanding that you use a metric to measure certain network > quantities, e.g. throughput, set up delays, etc. > In a second step you'd use these measured metrics to determine whether the > systems under test meet your previously expected or defined quality > parameters. If not, you'd make changes to the components or network > design, > measure again, and compare the previous set of data to the one after the > change and see whether you have improved and are moving into the right > direction. > > You seem to be coming from the other end, i.e. tweak the input parameters > to > suit your system under test. [sunwq] OK, I get your point here. What you said is true. I had thought by saying ``a usable lambda_m", you are saying that a lambda_m that is *best achievable* by the DUT/SUT. If you are expecting certain performance from the DUT/SUT, then very likely you will have in mind an "expected" lambda_m, won't you? The methodology here does not suggest how to select lambda_m. This in fact can provide flexibility. Either the test will use an expected lambda_m, or tune the lambda_m that reflects the best system performance, depending on the purpose of the test. Does this make sense? > > [RS8] > How to compare metrics that have been generated by two different Poisson > processes, i.e. e.g. two different arrival rates? > [sunwq] > No way. Do we need to do that anyway? > > [RS8a] > Pls also see my comments above. > If you cannot compare two sets of measurements how will you then arrive at > an objective study of the network components under test? [sunwq] Maybe I misunderstood you again here. The Poisson arrival rate is an approximation of real traffic load. Of course it is worthwhile to find out how DUT/SUT performs when traffic load changes. But what else do you expect by comparing two data sets from different traffic load? Did I miss something here? > > > [RS9] > These sample tuples are dependent on the arrival rate Lambda_m. How to > deduct this input parameter from a resulting sample? > For instance, if two result sample sets are given, how can we be sure that > they can be compared? > [sunwq] > My understanding of the sample is that it is a sequence of singleton > values > obtained under a set of parameters. The parameters, such as Lambda_m, are > not visible in the sample. However, when reporting the sample, the > parameters must also be reported. > > [RS9a] > OK. Noted and understood. [sunwq] Great, thanks. > > > [RS10] > A superposition of two different Poisson processes is being used here. > How do you map the resulting RESV messages to the correct process, i.e. > how > to differentiate e.g. a late RESV message resulting from third PATH > message > in fourth invocation from an early RESV message solicited by first PATH > message in fourth invocation? > [sunwq] > RESV messages can be differentiated by either MSG ID or non-overlapping > LSP > ID, or combined. The RSVP-TE protocol does that automatically. So I > believe > this is an RSVP-TE implementation issue, rather than a testing methodology > issue. > > [RS10a] > OK. Noted and understood. [sunwq] Great, thanks. > > [RS11] > ???? > [sunwq] > Well, the full text is like: "The percentile of Metric is defined as: > given > a metric and a percent X between 0% and 100%, the Xth percentile of all > the > dT values in the sample." Would it help a bit if sentence is replaced by > the > > full text above? > Please be noted that this text is inherited from the IPPM docs (See RFC > 2681, page 16, 4.1). > > [RS11a] > The way it was written in the doc it was just a fragment of a sentence. > Unfortunately RFC 2681, page 16, 4.1. also does not give a proper > definition, as it too is just a fragmented sentence. > RFC 2330, p.26, 11.3 gives a better explanation, as: > "We will use the term 'percentile' to refer to the smallest value of x for > which the empirical distribution function is greater or equal a given > percentage" [sunwq] Well, to tell the truth I was troubled by this too. Later I realized that one reason that might lead to such a definition is that the authors of 2681 had assumed that ``percentile" is a ``well known" term, and does not inheriting its definition from RFC 2330. And quite surprisingly wikipedia says "there is no standard definition of percentile." But wikipedia also says ``however all definitions yield similar results when the number of observations is large." I will replace the original example with the following to avoid such a dilemma (of conflicting calculations in two RFCs). ============original text=============== Example: suppose we take a sample and the results are: Stream1 = < <T1, 100 msec>, <T2, 110 msec>, <T3, undefined>, <T4, 90 msec>, <T5, 500 msec> > Then the 50th percentile would be 110 msec, since 90 msec and 100 msec are smaller, and 110 and 500 msec are larger (undefined values are not counted in). ============end of original text======== ============new text=============== Example: suppose we take a sample and the results are: Stream1 = < <T1, 100 msec>, <T2, 110 msec>, <T3, undefined>, <T4, 90 msec>, <T5, 500 msec>, <T6, 350msec> > Then the 60th percentile would be 110 msec, since 90 msec and 100 msec are smaller, and 110, 350 and 500 msec are larger (undefined values are not counted in). A more detailed definition of percentile may be found in RFC 2330. ============end of new text======== > [RS12] > As stipulated in RFC2330 the EDF is defined as a function F(x) which for > any > > x gives the fractional proportion of the total measurements that were <= > x. > (see RFC2330 11.3 p26) > Hence F(90)=0.25, F(100)=0.5,F(110)=0.75, F(500)=1 and therefore the 50th > percentile is defined as 100 as opposed to 110. > [sunwq] > Yes you are right with regard to RFC 2330. But we have used exactly the > same > > example as in RFC 2681 (page 16, section 4.1). The good thing is that the > difference of the two calculations is very small when the number of > measurement is large. Change needed? > > [RS12a] > As outlined above RFC 2330 as opposed to RFC2681 gives a proper definition > of percentile. Therefore we should stick to RFC2330 and change > accordingly. > > It should be noted though, that also RFC2330 defines percentile > differently > than it is done in the statistical literature, but we can live with this, > as > it makes the definition easier to apply to measurement samples. Good point, thanks for explaining. I hope the change above is satisfactory for you. Thanks and best regards, Weiqiang > > [------------------End of new comments---------------------] > > > And I realized that the points you have raised covered the main part of > the > text. We would expect your further comments regarding our responses. > Otherwise we will assume that you are happy with it and will proceed to > change the text based on the revision you sent in your previous email :) > > Thank you again for your time in reviewing the draft so carefully. > > Best regards, > Weiqiang > > -- > Weiqiang Sun > Shanghai Jiao Tong University > http://front.sjtu.edu.cn/~sunwq/ > > -------------------------------------------------- > From: "Rschrage" <rschrage@schrageconsult.net> > Sent: Wednesday, August 19, 2009 9:57 PM > To: "'Weiqiang Sun'" <sunwq@MIT.EDU>; "'Lou Berger'" <lberger@labn.net>; > "'Henk Uijterwaal'" <henk@ripe.net> > Cc: <ccamp@ietf.org>; <zhangguoying@mail.ritt.com.cn>; "'IETF IPPM WG'" > <ippm@ietf.org> > Subject: Re: [CCAMP] [ippm] [Fwd: IPPM expert review request] > >> Hi all, >> >> pls find attached a further updated edition of previous revised doc >> draft-ietf-ccamp-lsp-dppm-06. >> >> I believe this is as far as we can go for the moment before a mutual >> satisfactory understanding has been reached and a new draft has been >> issued >> by the authors. >> >> Again, pls feel free to comment. >> >> Many thanks. >> Brgds. >> Reinhard Schrage >> Tel: +49 (0) 5137 909540 >> Mobile: +49 (0) 172 26.36.046 >> reinhard@schrageconsult.com >> >> -----Original Message----- >> From: ippm-bounces@ietf.org [mailto:ippm-bounces@ietf.org] On Behalf Of >> Weiqiang Sun >> Sent: Mittwoch, 19. August 2009 09:28 >> To: Rschrage; 'Lou Berger'; 'Henk Uijterwaal' >> Cc: ccamp@ietf.org; zhangguoying@mail.ritt.com.cn; 'BRUNGARD, DEBORAH A, >> ATTLABS'; 'IETF IPPM WG' >> Subject: Re: [ippm] [Fwd: IPPM expert review request] >> >> More responses below: >> >> RS4. >> Should be more specific as to why this metric is no simple function of >> single uni-directional LSP Setup Delay metric, i.e. why can it not be >> deduced from single LSP Setup Delay? >> [sunwq] >> This point is raised regarding the sentence "The time needed to setup a >> large number of LSPs during a short time period can not be deduced by >> single >> >> LSP setup delay." >> One reason is that when a large number of LSPs in being setup during a >> short >> >> period of time, the control plane may be crowded or even overwhelmed by >> the >> large number of signaling and routing messages. This may significantly >> slow >> down the processing of a particular signaling message, which will results >> in >> >> longer LSP provisioning delays. >> Are you suggesting that we add similar explanations to the document? We'd >> love to do that if you feel it is necessary. >> >> RS5. >> The current definition only addresses multiple uni-directional LSP setups >> between two nodes ID0 and ID1. What about the the case when ID0 is to >> setup >> multiple uni-directional LSP setups to different egress nodes ID1, ID2, >> ID3,.,IDN? >> [sunwq] >> Yes, it is a useful case. Likewise you would expect a case to establish a >> varied number of (instead of one) LSPs to a set of destinations. Our >> suggestion is that this case is approached by integrating the defined >> methodologies of single/multiple LSPs provisioning delay, in the form of >> a >> particular testing case, rather than a formal testing methodology. This >> is >> in fact a trade-off between complexity and accuracy. >> >> Best regards, >> Weiqiang >> >> -- >> Weiqiang Sun >> Shanghai Jiao Tong University >> http://front.sjtu.edu.cn/~sunwq/ >> >> -------------------------------------------------- >> From: "Weiqiang Sun" <sunwq@mit.edu> >> Sent: Wednesday, August 19, 2009 2:48 PM >> To: "Rschrage" <rschrage@schrageconsult.net>; "'Lou Berger'" >> <lberger@labn.net>; "'Henk Uijterwaal'" <henk@ripe.net> >> Cc: <zhangguoying@mail.ritt.com.cn>; "'BRUNGARD, DEBORAH A, ATTLABS'" >> <dbrungard@att.com>; "'IETF IPPM WG'" <ippm@ietf.org>; <ccamp@ietf.org> >> Subject: Re: [ippm] [Fwd: IPPM expert review request] >> >>> Hi Reinhard, >>> >>> Many thanks for doing the careful review. We appreciate your many >>> wording >>> suggestions. We will go through these one by one and adopt those we feel >>> appropriate. In this email I will not list all the comments since these >>> will not lead to major technical changes. >>> >>> Below I try to respond to the points you raised in the revision. >>> >>> RS1. >>> This would imply that the metric may produce a range of several values >>> at >>> one time with the minimum value to be taken from that range. >>> I believe what the authors are trying to say is, that the LSP Setup >>> Delay >>> Metric provides a lower bound on all possible LSP Delay Metrics of >>> actually instantiated LSPs, or in other words: an actual LSP Delay >>> Metric >>> cannot get smaller than an LSP Setup Delay Metric. >>> [sunwq] >>> This point is raised regarding the motivation of the sigleton definition >>> of Single Uni-directional LSP Setup Delay (ie. 4.1, para 2). We are >>> trying >> >>> to say that the minimum value of this metric reflects the (likely) >>> single >>> lsp setup delay when the control plane is lightly loaded. The formal >>> definition of the minimum value is in "14.1 The minimum of metric" >>> In fact we have inherited this from the IPPM documents (see RFC 2681, >>> page >> >>> 2 section 1.1 bullet 3). >>> >>> RS2. >>> How? Should there be a methodology be defined for this? >>> [sunwq] >>> This point is raised regarding the sentense in our methodologies >>> sections - "Make sure that the network has enough resource to set up the >>> requested LSP." >>> We have assumed that test personnel should have adequate expertise in >>> allocating enough resources for the testing. The allocation of resources >>> for the testing purpose can be very test specific and we believe it is >>> quite outside the scope of this document. >>> >>> RS3. >>> Too vague? As both timestamps are to be taken on the same ingress node, >>> they should be taken at the same application/network level. >>> [sunwq] >>> This point is raised regarding the sentense "If the corresponding RESV >>> message arrives within a reasonable period of time, take the timestamp >>> (T2) as soon as possible upon receipt of the message." >>> Yes, ideally the timestamp should be taken immediately after the >>> reception >> >>> of the RESV message. The wording "as soon as possible" is used to allow >>> for some flexibility when RESV message needs to be propagated to certain >>> modules where timestamp-taking is most appropriate. >>> And again, this text is inherited from the IPPM documents (see RFC 2681 >>> page 7, bullet 3). >>> >>> Thanks again and looking forward to your further comments and revisions. >>> >>> Weiqiang >>> >>> -- >>> Weiqiang Sun >>> Shanghai Jiao Tong University >>> http://front.sjtu.edu.cn/~sunwq/ >>> >>> -------------------------------------------------- >>> From: "Rschrage" <rschrage@schrageconsult.net> >>> Sent: Tuesday, August 18, 2009 8:23 PM >>> To: "'Lou Berger'" <lberger@labn.net>; "'Henk Uijterwaal'" >>> <henk@ripe.net> >>> Cc: <zhangguoying@mail.ritt.com.cn>; "'BRUNGARD, DEBORAH A, ATTLABS'" >>> <dbrungard@att.com>; <sunwq@mit.edu>; "'IETF IPPM WG'" <ippm@ietf.org>; >>> <ccamp@ietf.org> >>> Subject: RE: [ippm] [Fwd: IPPM expert review request] >>> >>>> Hello, >>>> >>>> sorry for late response- >>>> >>>> Nevertheless here is a first edition of my revision of doc >>>> draft-ietf-ccamp-lsp-dppm-06. >>>> >>>> The doc has been saved in Word 2003 format to easily track changes. >>>> I am using Kaspersky Anti Virus 2010 software so I trust there should >>>> be >>>> no >>>> unpleasant surprises when downloading the attached word doc. >>>> >>>> >>>> I thought it would be advisable to first address the suggested comments >>>> and >>>> questions upto and including the uni-directional LSP setup delay metric >>>> and >>>> after that to continue with the bi-directional and sample definitions >>>> as >>>> covered in the doc. >>>> >>>> Please let me have your thoughts on this. >>>> >>>> Many thanks. >>>> Reinhard Schrage >>>> Tel: +49 (0) 5137 909540 >>>> Mobile: +49 (0) 172 26.36.046 >>>> reinhard@schrageconsult.com >>>> >>>> -----Original Message----- >>>> From: ippm-bounces@ietf.org [mailto:ippm-bounces@ietf.org] On Behalf Of >>>> Lou >>>> Berger >>>> Sent: Dienstag, 28. Juli 2009 14:59 >>>> To: Henk Uijterwaal >>>> Cc: zhangguoying@mail.ritt.com.cn; BRUNGARD, DEBORAH A, ATTLABS; >>>> sunwq@mit.edu; IETF IPPM WG >>>> Subject: Re: [ippm] [Fwd: IPPM expert review request] >>>> >>>> Hank/Reinhard, >>>> >>>> Thank you very much for undertaking this review. Please cc >>>> ccamp@ietf.org on any comments you may have. >>>> >>>> Lou >>>> >>>> On 7/28/2009 6:47 AM, Henk Uijterwaal wrote: >>>>> Lou, others, >>>>> >>>>>> We received the request for a review of a document currently under >>>>>> discussion in the CCAMP WG, please see below. Is there anybody who >>>>>> has time to do this review in the near future? >>>>> >>>>> Reinhard Schrage (rschrage@schrageconsult.net) has voluntered to do >>>>> this. >>>>> >>>>> Reinhard: please post anything you find to both the list and the >>>>> authors. >>>>> And thank you for doing this. >>>>> >>>>> Henk >>>>> >>>>>> >>>>>> Matt & Henk >>>>>> >>>>>> -------- Original Message -------- >>>>>> Subject: IPPM expert review request >>>>>> Date: Fri, 24 Jul 2009 15:57:41 -0400 >>>>>> From: Lou Berger <lberger@labn.net> >>>>>> To: ippm-chairs@tools.ietf.org >>>>>> CC: Brungard, Deborah A, ALABS <dbrungard@att.com>, sunwq@mit.edu, >>>>>> zhangguoying <zhangguoying@mail.ritt.com.cn> >>>>>> >>>>>> Hi, >>>>>> We, the CCAMP WG chairs, would like to request that the IPPM WG >>>>>> review >>>>>> a draft that is progressing through the CCAMP WG. This work applies >>>>>> IPPM approaches to GMPLS. The document we'd like reviewed is >>>>>> available at: >>>>>> >>>>>> http://tools.ietf.org/html/draft-ietf-ccamp-lsp-dppm-06 >>>>>> >>>>>> Is this acceptable? Can you undertake this review? Alternatively, >>>>>> we >>>>>> can just last call the document in your WG (it has already passed >>>>>> CCAMP >>>>>> WG LC). >>>>>> >>>>>> Thank you, >>>>>> Lou (and Deborah) >>>>>> >>>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> ippm mailing list >>>> ippm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ippm >>>> >> _______________________________________________ >> ippm mailing list >> ippm@ietf.org >> https://www.ietf.org/mailman/listinfo/ippm >> > > > >> _______________________________________________ >> CCAMP mailing list >> CCAMP@ietf.org >> https://www.ietf.org/mailman/listinfo/ccamp >> > >
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Lou Berger
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Rschrage
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Thomas D. Nadeau
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Lou Berger
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Thomas D. Nadeau
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Rschrage
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Rschrage
- Re: [CCAMP] [ippm] [Fwd: IPPM expert review reque… Weiqiang Sun