Re: [CCAMP] 2nd Working group last call: draft-ietf-ccamp-lsp-dppm

[Lou Berger <[e-mail]>]

Weiqiang,
	Please see below.

On 9/15/2009 1:23 AM, Weiqiang Sun wrote:
> Hi Lou,
> 
> Thank you for the comments. Please see below.
> 
> Best regards,
> Weiqiang
> 
> -- 
> Weiqiang Sun
> Shanghai Jiao Tong University
> http://front.sjtu.edu.cn/~sunwq/
> 
> --------------------------------------------------
> From: "Lou Berger" <[e-mail]>
> Sent: Saturday, September 12, 2009 6:08 AM
> To: <[e-mail]>
> Cc: <[e-mail]>
> Subject: Re: [CCAMP] 2nd Working group last call: draft-ietf-ccamp-lsp-dppm
> 
>> Authors,
>>
>> The following are LC comments.  There are a couple of followup points
>> from the 1st LC and then a slew of non-technical/editorial comments.
>>
>> - I'm happy to see additional RFC2119 directives to match your
>>  desired document track.  Don't you think it would be worthwhile
>>  to add directives on what reporting is needed for the metrics
>>  actually being measured?  As I read it, only bounds are reported.
>>  This really doesn't seem to make sense.  Am I missing something?
>>
> Maybe we can change the text into:
> ``In practice, the upper bound should be chosen carefully and the value
> MUST
> be reported along with the metric itself."
> Do you think it is helpful?

How about (section 4-8):
   In practice, the upper bound should be chosen carefully.

o  All collected metrics together with the selected upper bound MUST
   be reported.

Section 9, 10, 11,12 ,13 :

o  All collected metrics together with the selected parameters MUST
   be reported.

> 
>> - I think Section 14.4 addresses much of my earlier comment on
>>  "undefined" values.  I still have a couple of additional points:
>>
>>   Don't you think it would be worthwhile to also report the Success
>>   Count as an integer?  (as percentages can yield rounding errors.)
> Please be noted that we already have the Failure Count(14.4.1). Success
> Count can be easily deduced from: ``the number of all values in a sample -
> Failure Count". Reporting the Success Count seems a bit redundant,
> when the Failure count is far more important and necessary.

I don't see where the total number of requests in a sample is described.
 If it is, then I agree that Success Count isn't needed.

> 
>>
>>   As part of the rfc2119 language you're adding related to the
>>   reporting of metrics defined in section 14, I suggest that you
>>   highlight the easily missed point of "we only consider the
>>   finite values" by including "success" as part of the metric
>>   name.  I'm not sure on exact phrasing as this will depend on
>>   how you write the language.
> Since the undefined values are properly treated  in section 14 using the
> rfc2119 language, maybe we can just drop
> the sentence "In the following discussion, we only consider the finite
> values." How do you think?
> 

If total or success count are added, this is fine.

>>
>>   Is there a reason that you included Minimum and Median in
>>   Section 14, but not maximum?
> Minimum and Median values are generally more stable than maximum in a large
> measurement and can provide valuable sights to the DUT/SUT.

Sure, but maximums provide an important view into what can be expected
from the system.  Without the max value there is no way to understand
worst case behavior. I think it's worth including.

> We do have a bunch of other statistics that are not yet included in the
> document. What we can offer is a suggested sub-set, rather than a whole
> set.
> The current selection is based on rfc2679 and rfc2681.
> 

well, seems like you should add a caveate along the lines of 2679, i.e.,
"These statistics are offered mostly to be illustrative of what could be
done."

>>
>> General comments:
> Many thanks for the comments below. We will make the changes accordingly.
> 

Thanks,
Lou