Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]

["Rschrage" <[e-mail]>]

Hi Weiqiang,

sorry for late reply-

My responses/comments follow below your points.

We seem to be coming from different view points

Brgds.
Reinhard Schrage
Tel: [phone]
Mobile: [phone]
[e-mail]

-----Original Message-----
From: Weiqiang Sun [mailto:[e-mail]] 
Sent: Freitag, 21. August 2009 08:59
To: Rschrage; 'Lou Berger'; 'Henk Uijterwaal'
Cc: [e-mail]; [e-mail]; 'IETF IPPM WG'
Subject: Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]

Hi Reinhard,

Thanks for sending this part. See below for my responses.

[RS7]
Is there any help or suggestion on how to actually compute a usable 
Lambda_m?
[sunwq]
No. Depending on the performance of the implementation, the lambda_m can be 
very different, even under the same topology. A proper value of Lambda_m can

only be tuned in the testing.

[RS7a]
I am a bit at loss here.

It was my understanding that you use a metric to measure certain network
quantities, e.g. throughput, set up delays, etc.
In a second step you'd use these measured metrics to determine whether the
systems under test meet your previously expected or defined quality
parameters. If not, you'd make changes to the components or network design,
measure again, and compare the previous set of data to the one after the
change and see whether you have improved and are moving into the right
direction.

You seem to be coming from the other end, i.e. tweak the input parameters to
suit your system under test.

[RS8]
How to compare metrics that have been generated by two different Poisson 
processes, i.e. e.g. two different arrival rates?
[sunwq]
No way. Do we need to do that anyway?

[RS8a]
Pls also see my comments above.
If you cannot compare two sets of measurements how will you then arrive at
an objective study of the network components under test?



[RS9]
These sample tuples are dependent on the arrival rate Lambda_m. How to 
deduct this input parameter from a resulting sample?
For instance, if two result sample sets are given, how can we be sure that 
they can be compared?
[sunwq]
My understanding of the sample is that it is a sequence of singleton values 
obtained under a set of parameters. The parameters, such as Lambda_m, are 
not visible in the sample. However, when reporting the sample, the 
parameters must also be reported.

[RS9a]
OK. Noted and understood.


[RS10]
A superposition of two different Poisson processes is being used here.
How do you map the resulting RESV messages to the correct process, i.e. how 
to differentiate e.g. a late RESV message resulting from third PATH message 
in fourth invocation from an early RESV message solicited by first PATH 
message in fourth invocation?
[sunwq]
RESV messages can be differentiated by either MSG ID or non-overlapping LSP 
ID, or combined. The RSVP-TE protocol does that automatically. So I believe 
this is an RSVP-TE implementation issue, rather than a testing methodology 
issue.

[RS10a]
OK. Noted and understood.

[RS11]
????
[sunwq]
Well, the full text is like: "The percentile of Metric is defined as: given 
a metric and a percent X between 0% and 100%, the Xth percentile of all the 
dT values in the sample." Would it help a bit if sentence is replaced by the

full text above?
Please be noted that this text is inherited from the IPPM docs (See RFC 
2681, page 16, 4.1).

[RS11a]
The way it was written in the doc it was just a fragment of a sentence.
Unfortunately RFC 2681, page 16, 4.1. also does not give a proper
definition, as it too is just a fragmented sentence. 
RFC 2330, p.26, 11.3 gives a better explanation, as:
"We will use the term 'percentile' to refer to the smallest value of x for
which the empirical distribution function is greater or equal a given
percentage"

[RS12]
As stipulated in RFC2330 the EDF is defined as a function F(x) which for any

x gives the fractional proportion of the total measurements that were <= x. 
(see RFC2330 11.3 p26)
Hence F(90)=0.25, F(100)=0.5,F(110)=0.75, F(500)=1 and therefore the 50th 
percentile is defined as 100 as opposed to 110.
[sunwq]
Yes you are right with regard to RFC 2330. But we have used exactly the same

example as in RFC 2681 (page 16, section 4.1). The good thing is that the 
difference of the two calculations is very small when the number of 
measurement is large. Change needed?

[RS12a]
As outlined above RFC 2330 as opposed to RFC2681 gives a proper definition
of percentile. Therefore we should stick to RFC2330 and change accordingly.

It should be noted though, that also RFC2330 defines percentile differently
than it is done in the statistical literature, but we can live with this, as
it makes the definition easier to apply to measurement samples.

[------------------End of new comments---------------------]


And I realized that the points you have raised covered the main part of the 
text. We would expect your further comments regarding our responses. 
Otherwise we will assume that you are happy with it and will proceed to 
change the text based on the revision you sent in your previous email :)

Thank you again for your time in reviewing the draft so carefully.

Best regards,
Weiqiang

--
Weiqiang Sun
Shanghai Jiao Tong University
http://front.sjtu.edu.cn/~sunwq/

--------------------------------------------------
From: "Rschrage" <[e-mail]>
Sent: Wednesday, August 19, 2009 9:57 PM
To: "'Weiqiang Sun'" <[e-mail]>; "'Lou Berger'" <[e-mail]>; 
"'Henk Uijterwaal'" <[e-mail]>
Cc: <[e-mail]>; <[e-mail]>; "'IETF IPPM WG'" 
<[e-mail]>
Subject: Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]

> Hi all,
>
> pls find attached a further updated edition of previous revised doc
> draft-ietf-ccamp-lsp-dppm-06.
>
> I believe this is as far as we can go for the moment before a mutual
> satisfactory understanding has been reached and a new draft has been 
> issued
> by the authors.
>
> Again, pls feel free to comment.
>
> Many thanks.
> Brgds.
> Reinhard Schrage
> Tel: [phone]
> Mobile: [phone]
> [e-mail]
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of
> Weiqiang Sun
> Sent: Mittwoch, 19. August 2009 09:28
> To: Rschrage; 'Lou Berger'; 'Henk Uijterwaal'
> Cc: [e-mail]; [e-mail]; 'BRUNGARD, DEBORAH A,
> ATTLABS'; 'IETF IPPM WG'
> Subject: Re: [ippm] [Fwd: IPPM expert review request]
>
> More responses below:
>
> RS4.
> Should be more specific as to why this metric is no simple function of
> single uni-directional LSP Setup Delay metric, i.e. why can it not be
> deduced from single LSP Setup Delay?
> [sunwq]
> This point is raised regarding the sentence "The time needed to setup a
> large number of LSPs during a short time period can not be deduced by 
> single
>
> LSP setup delay."
> One reason is that when a large number of LSPs in being setup during a 
> short
>
> period of time, the control plane may be crowded or even overwhelmed by 
> the
> large number of signaling and routing messages. This may significantly 
> slow
> down the processing of a particular signaling message, which will results 
> in
>
> longer LSP provisioning delays.
> Are you suggesting that we add similar explanations to the document? We'd
> love to do that if you feel it is necessary.
>
> RS5.
> The current definition only addresses multiple uni-directional LSP setups
> between two nodes ID0 and ID1. What about the the case when ID0 is to 
> setup
> multiple uni-directional LSP setups to different egress nodes ID1, ID2,
> ID3,.,IDN?
> [sunwq]
> Yes, it is a useful case. Likewise you would expect a case to establish a
> varied number of (instead of one) LSPs to a set of destinations. Our
> suggestion is that this case is approached by integrating the defined
> methodologies of single/multiple LSPs provisioning delay, in the form of a
> particular testing case, rather than a formal testing methodology. This is
> in fact a trade-off between complexity and accuracy.
>
> Best regards,
> Weiqiang
>
> --
> Weiqiang Sun
> Shanghai Jiao Tong University
> http://front.sjtu.edu.cn/~sunwq/
>
> --------------------------------------------------
> From: "Weiqiang Sun" <[e-mail]>
> Sent: Wednesday, August 19, 2009 2:48 PM
> To: "Rschrage" <[e-mail]>; "'Lou Berger'"
> <[e-mail]>; "'Henk Uijterwaal'" <[e-mail]>
> Cc: <[e-mail]>; "'BRUNGARD, DEBORAH A, ATTLABS'"
> <[e-mail]>; "'IETF IPPM WG'" <[e-mail]>; <[e-mail]>
> Subject: Re: [ippm] [Fwd: IPPM expert review request]
>
>> Hi Reinhard,
>>
>> Many thanks for doing the careful review. We appreciate your many wording
>> suggestions. We will go through these one by one and adopt those we feel
>> appropriate. In this email I will not list all the comments since these
>> will not lead to major technical changes.
>>
>> Below I try to respond to the points you raised in the revision.
>>
>> RS1.
>> This would imply that the metric may produce a range of several values at
>> one time with the minimum value to be taken from that range.
>> I believe what the authors are trying to say is, that the LSP Setup Delay
>> Metric provides a lower bound on all possible LSP Delay Metrics of
>> actually instantiated LSPs, or in other words: an actual LSP Delay Metric
>> cannot get smaller than an LSP Setup Delay Metric.
>> [sunwq]
>> This point is raised regarding the motivation of the sigleton definition
>> of Single Uni-directional LSP Setup Delay (ie. 4.1, para 2). We are 
>> trying
>
>> to say that the minimum value of this metric reflects the (likely) single
>> lsp setup delay when the control plane is lightly loaded. The formal
>> definition of the minimum value is in "14.1 The minimum of metric"
>> In fact we have inherited this from the IPPM documents (see RFC 2681, 
>> page
>
>> 2 section 1.1 bullet 3).
>>
>> RS2.
>> How? Should there be a methodology be defined for this?
>> [sunwq]
>> This point is raised regarding the sentense in our methodologies
>> sections - "Make sure that the network has enough resource to set up the
>> requested LSP."
>> We have assumed that test personnel should have adequate expertise in
>> allocating enough resources for the testing. The allocation of resources
>> for the testing purpose can be very test specific and we believe it is
>> quite outside the scope of this document.
>>
>> RS3.
>> Too vague? As both timestamps are to be taken on the same ingress node,
>> they should be taken at the same application/network level.
>> [sunwq]
>> This point is raised regarding the sentense "If the corresponding RESV
>> message arrives within a reasonable period of time, take the timestamp
>> (T2) as soon as possible upon receipt of the message."
>> Yes, ideally the timestamp should be taken immediately after the 
>> reception
>
>> of the RESV message. The wording "as soon as possible" is used to allow
>> for some flexibility when RESV message needs to be propagated to certain
>> modules where timestamp-taking is most appropriate.
>> And again, this text is inherited from the IPPM documents (see RFC 2681
>> page 7, bullet 3).
>>
>> Thanks again and looking forward to your further comments and revisions.
>>
>> Weiqiang
>>
>> --
>> Weiqiang Sun
>> Shanghai Jiao Tong University
>> http://front.sjtu.edu.cn/~sunwq/
>>
>> --------------------------------------------------
>> From: "Rschrage" <[e-mail]>
>> Sent: Tuesday, August 18, 2009 8:23 PM
>> To: "'Lou Berger'" <[e-mail]>; "'Henk Uijterwaal'" 
>> <[e-mail]>
>> Cc: <[e-mail]>; "'BRUNGARD, DEBORAH A, ATTLABS'"
>> <[e-mail]>; <[e-mail]>; "'IETF IPPM WG'" <[e-mail]>;
>> <[e-mail]>
>> Subject: RE: [ippm] [Fwd: IPPM expert review request]
>>
>>> Hello,
>>>
>>> sorry for late response-
>>>
>>> Nevertheless here is a first edition of my revision of doc
>>> draft-ietf-ccamp-lsp-dppm-06.
>>>
>>> The doc has been saved in Word 2003 format to easily track changes.
>>> I am using Kaspersky Anti Virus 2010 software so I trust there should be
>>> no
>>> unpleasant surprises when downloading the attached word doc.
>>>
>>>
>>> I thought it would be advisable to first address the suggested comments
>>> and
>>> questions upto and including the uni-directional LSP setup delay metric
>>> and
>>> after that to continue with the bi-directional and sample definitions as
>>> covered in the doc.
>>>
>>> Please let me have your thoughts on this.
>>>
>>> Many thanks.
>>> Reinhard Schrage
>>> Tel: [phone]
>>> Mobile: [phone]
>>> [e-mail]
>>>
>>> -----Original Message-----
>>> From: [e-mail] [mailto:[e-mail]] On Behalf Of
>>> Lou
>>> Berger
>>> Sent: Dienstag, 28. Juli 2009 14:59
>>> To: Henk Uijterwaal
>>> Cc: [e-mail]; BRUNGARD, DEBORAH A, ATTLABS;
>>> [e-mail]; IETF IPPM WG
>>> Subject: Re: [ippm] [Fwd: IPPM expert review request]
>>>
>>> Hank/Reinhard,
>>>
>>> Thank you very much for undertaking this review.  Please cc
>>> [e-mail] on any comments you may have.
>>>
>>> Lou
>>>
>>> On 7/28/2009 6:47 AM, Henk Uijterwaal wrote:
>>>> Lou, others,
>>>>
>>>>> We received the request for a review of a document currently under
>>>>> discussion in the CCAMP WG, please see below.  Is there anybody who
>>>>> has time to do this review in the near future?
>>>>
>>>> Reinhard Schrage ([e-mail]) has voluntered to do
>>>> this.
>>>>
>>>> Reinhard: please post anything you find to both the list and the
>>>> authors.
>>>> And thank you for doing this.
>>>>
>>>> Henk
>>>>
>>>>>
>>>>> Matt & Henk
>>>>>
>>>>> -------- Original Message --------
>>>>> Subject: IPPM expert review request
>>>>> Date: Fri, 24 Jul 2009 15:57:41 -0400
>>>>> From: Lou Berger <[e-mail]>
>>>>> To: [e-mail]
>>>>> CC: Brungard, Deborah A, ALABS <[e-mail]>, [e-mail],
>>>>> zhangguoying <[e-mail]>
>>>>>
>>>>> Hi,
>>>>>     We, the CCAMP WG chairs, would like to request that the IPPM WG
>>>>> review
>>>>> a draft that is progressing through the CCAMP WG.  This work applies
>>>>> IPPM approaches to GMPLS.  The document we'd like reviewed is
>>>>> available at:
>>>>>
>>>>> http://tools.ietf.org/html/draft-ietf-ccamp-lsp-dppm-06
>>>>>
>>>>> Is this acceptable?  Can you undertake this review?  Alternatively, we
>>>>> can just last call the document in your WG (it has already passed 
>>>>> CCAMP
>>>>> WG LC).
>>>>>
>>>>> Thank you,
>>>>> Lou (and Deborah)
>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> ippm mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/ippm
>>>
> _______________________________________________
> ippm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ippm
>



> _______________________________________________
> CCAMP mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ccamp
>