Re: [PSAMP] PSAMP-INFO IE realtiveError

["Zseby, Tanja" <[e-mail]>]

Hi Paul,

see comments below.

> -----Original Message-----
> From: Paul Aitken [mailto:[e-mail]]
> Sent: Thursday, August 07, 2008 10:57 AM
> To: Zseby, Tanja
> Cc: Benoit Claise; psamp; Juergen Quittek
> Subject: Re: PSAMP-INFO IE realtiveError
> 
> Tanja,
> 
> > Hi Benoit and Paul,
> >
> >
> >
> > here my suggestions for clarification of the error IEs in
PSAMP-INFO.
> >
> > -    I suggest to rename the fixedError to absoluteError
> 
> Agreed.
> 
> 
> > -    I suggest to introduce CI limits and level to also report
> > estimation errors
> >
> > -    If it is still possible I would suggest to make a few small
> changes
> > in PSAMP-PROTO for consistency.
> >
> > -    Upper and lower CI limits can be also specified as provided
> > absolute or relative limits. So we could also add 2 more IEs (for
the
> > relative CI limits)
> >
> >
> >
> > New description of IEs:
> >
> >
> >
> > absoluteError
> >
> > This Information Element specifies the maximum possible measurement
> > error of the reported value for a given Information Element. The
> 
> We should indicate how to connect the *Error values to specific
fields,
> eg by using an option with the specific field as scope. Else, someone
> may put the *Error elements adjacent to the relative fields in the
data
> record - which could work, but is open to misinterpretation.

Maybe I don't understand the comment correctly. Isnt the usage explained
in the example in PSAMP-PROTO? The error can also be applied to relative
values.

> 
> 
> > absoluteError has the same unit as the information element it is
> > associated to. The real value of the metric can differ by
> > absoluteError
> 
> "with" ------^^

? Where to put the with?

> 
> > (positive or negative) from the measured value. This information
> > element provides only the error for measured values. If an
> information
> > element contains an estimated values (from sampling) the confidence
> > boundaries and confidence level have to be provided instead.
> 
> I would name the IEs: "the confidence boundaries and confidence level
> have to be provided instead (with the upperCILimit, lowerCILimit and
> confidenceLevel).

o.k.

> 
> 
> > relativeError
> >
> > This Information Element specifies the maximum possible measurement
> > error of the reported value for a given Information Element as
> > percentage of the measured value. The real value of the metric can
> > differ by relativeError percent (positive or negative) from the
> > measured value. This information element provides only the error for
> > measured values. If an information element contains an estimated
> > values (from
> > sampling) the confidence boundaries and confidence level have to be
> > provided instead.
> 
> Again, as above, I would specifically name the IEs for this.


again o.k.

> 
> 
> > upperCILimit
> >
> > This Information Element specifies the upper limit of a confidence
> > interval. It is used to provide an accuracy statement for an
> estimated
> > value. The confidence limits define the range in which the real
value
> > is assumed to be with a certain probability p. Confidence limits
> > always need to be associated with a confidence level that defines
> this
> > probability p. Please note that a confidence interval only provides
a
> > probability that the real values lies within the limits. That means
> > the real value can lie outside the confidence limits.
> >
> >
> >
> > lowerCILimit
> >
> > This Information Element specifies the lower limit of a confidence
> > interval. For further information see the description of
> upperCILimit.
> >
> >
> >
> > confidenceLevel
> >
> > This Information Element specifies the confidence level. It is used
> to
> > provide an accuracy statement for estimated values. The confidence
> > level provides the probability p with which the real value lies
> within
> > a given range. A confidence level always needs to be associated with
> > confidence limits that define the range in which the real value is
> assumed to be.
> 
> We should specify that upperCILimit, lowerCILimit and confidenceLevel
> are all required, and what to do if too few of them are provided.
> 

Maybe just a sentence: "All three values (upperCILimit, lowerCILimit and
confidenceLevel) are necessary to provide an complete accuracy
statement."

I think the checking for the complete accuracy statement as out of scope
for IPFIX/PSAMP. I think this is something that the applications that
requires the statement should check. So I would consider no mandatory
action by collector.

> 
> > Changes to PSAMP-PROTO if still possible:
> >
> >
> >
> > -    Rename fixedError to absoluteError
> >
> > -    Slightly modify paragraph 2
> >
> > OLD:
> >
> > ...  The accuracy SHOULD be reported either with the fixedError
> > Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO>],
> > or with the relativeError Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO>].
> >
> > NEW:
> >
> > ... The accuracy for a measured information elelment SHOULD be
> reported
> > either with the fixedError Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO>],
> > or with the relativeError Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO>].
> > The accuracy for an estimated information element (from sampling)
> SHOULD
> > be reported with confidence limits and confidence level.[PSAMP-INFO]
> 
> Agreed. I'd also like to add something indicating how this can be
done,
> eg by using an option with the correct scope.

o.k. Maybe you can add a sentence for this?

> 
> 
> >
> >
> >
> > -    Remove the following paragraph (very important! Otherwise it
> would
> > lead to confusion):
> >
> > For example, the accuracy of an Information Element to estimate the
> > accuracy of a sampled flow, for which the unit would be specified in
> > octets, can be specified with the relativeError Information Element
> with
> > the octet units.  In this case, the error interval is the
Information
> > Element value +/- the value reported in the relativeError times the
> > reported Information Element value.
> >
> >
> >
> > -    Avoid the term error interval
> >
> > OLD:
> >
> > In this case, the error interval is the Information Element value
+/-
> > the value reported in the fixedError.
> >
> > NEW:
> >
> > In this case, the real values lies within the range of the
> Information
> > Element value +/- the value reported in the absoluteError.
> >
> >
> >
> >
> >
> > -    Remove the following paragraph (since absolute or relative
error
> > are just different representations I would not gain something if I
> > report both)
> >
> > Alternatively to reporting either the fixedError Information Element
> or
> > the relativeError Information Element in the Accuracy Report
> > Interpretation, both Information Elements MAY be present.  This
> scenario
> > could help in more complex situations where the system clock drifts,
> on
> > the top of having its own accuracy, during the duration of a
> measurement.
> 
> The intention was to say that the clock is 5 minutes slow, +/- 10
> seconds - so there's both an absolute error and a relative error.
> 

NOW I finally understand what you meant by fixedError initially !!
This I would not consider as error. Any fixed deviation I would rather
name "offset"...
If it is known couldn't you add it to the value and report the correct
time?
Or maybe check whether the NTP and TICTOC have a term for this...


Kind regards
Tanja

> > Sorry for the late comments, I was quite busy with PSAMP-TECH
> before...
> 
> Thanks for your input!
> 
> 
> --
> Paul Aitken
> Cisco Systems Ltd, Edinburgh, Scotland.
_______________________________________________
PSAMP mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/psamp