Re: [PSAMP] PSAMP-INFO IE realtiveError

["Zseby, Tanja" <[e-mail]>]

Hi Jürgen,

here my suggested changes to PSAMP-PROTO section 6.5.4 (see my initial email):

- Rename fixedError to absoluteError
- slightly modify paragraph 2
OLD:
...  The accuracy SHOULD be reported either with the fixedError Information Element [PSAMP-INFO <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO> ], or with the relativeError Information Element [PSAMP-INFO <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO> ].
NEW:
... The accuracy for a measured information elelment SHOULD be reported either with the fixedError Information Element [PSAMP-INFO <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO> ], or with the relativeError Information Element [PSAMP-INFO <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO> ]. The accuracy for an estimated information element (from sampling) SHOULD be reported with confidence limits and confidence level.[PSAMP-INFO]

- Remove the following paragraph (this I consider as most important!)
For example, the accuracy of an Information Element to estimate the accuracy of a sampled flow, for which the unit would be specified in octets, can be specified with the relativeError Information Element with the octet units.  In this case, the error interval is the Information Element value +/- the value reported in the relativeError times the reported Information Element value.

- Avoid the term error interval
OLD: 
In this case, the error interval is the Information Element value +/- the value reported in the fixedError.
NEW: 
In this case, the real values lies within the range of the Information Element value +/- the value reported in the absoluteError.
 
- Remove the following paragraph (since absolute or relative error are just different representations I would not gain something if I report both)

Alternatively to reporting either the fixedError Information Element or the relativeError Information Element in the Accuracy Report Interpretation, both Information Elements MAY be present.  This scenario could help in more complex situations where the system clock drifts, on the top of having its own accuracy, during the duration of a measurement.


Kind regards,
Tanja

> -----Original Message-----
> From: Juergen Quittek [mailto:[e-mail]]
> Sent: Friday, August 01, 2008 4:53 PM
> To: Zseby, Tanja; Benoit Claise
> Cc: IETF PSAMP Working Group
> Subject: Re: [PSAMP] PSAMP-INFO IE realtiveError
> 
> Tanja,
> 
> If you can suggest a new paragraph for PSAMP-PROTO and if the authors
> support it, I can ask Dan for his support and to forward it as an RFC
> Editor comment. This would be a much cleaner procedure than applying
> changes at AUTH48.  And if it got rejected, we would just follow
> Benoit's proposal.
> 
> Thanks,
> 
>     Juergen
> 
> 
> Am 01.08.08 10:28 schrieb "Tanja Zseby" unter
> <[e-mail]>:
> 
> > Hi Benoit,
> >
> > I think we need no new section in PSAMP-PROTO and can only introduce
> > the CI limits in PSAMP-INFO. We can stick with the example in
> > PSAMP-PROTO but need to clarify that the example is only for the
> > accuracy of measured values. That means mainly to remove the 2
> > sentences that refer to estimation error. It depends how much we can
> > still change in PSAMP-PROTO. Is removing 2 sentences o.k.?
> >
> > Kind regards
> > Tanja
> >
> >> -----Original Message-----
> >> From: [e-mail] [mailto:[e-mail]] On
> >> Behalf Of Benoit Claise
> >> Sent: Friday, August 01, 2008 12:46 AM
> >> To: Zseby, Tanja
> >> Cc: psamp; Juergen Quittek
> >> Subject: Re: [PSAMP] PSAMP-INFO IE realtiveError
> >>
> >> Hi Tanja,
> >>
> >>
> >> Hi Benoit and Paul,
> >>
> >>
> >>
> >> here my suggestions for clarification of the error IEs in PSAMP-
> >> INFO.
> >>
> >> -    I suggest to rename the fixedError to absoluteError
> >>
> >> No problem with that, but [PSAMP-PROTO] must follow otherwise we
> have
> > a
> >> problem Can we still change that in AUTH48 maybe?
> >>
> >>
> >>
> >>
> >> -    I suggest to introduce CI limits and level to also report
> >> estimation errors
> >>
> >> I'm wondering whether this is a good idea to add upperCILimit,
> >> lowerCILimit, and confidenceLevel at this stage.
> >> Because it implies that we need a complete new section in
> > [PSAMP-PROTO]
> >> (as opposed to just editorial change) similar to "Accuracy Report
> >> Interpretation" but for the accuracy statement for estimated value.
> >> Now, the simple solution is to add the information elements in
> PSAMP-
> >> INFO and don't discuss the accuracy statement for estimated value in
> >> [PSAMP-PROTO].
> >>
> >>
> >>
> >>
> >> -    If it is still possible I would suggest to make a few small
> >> changes in PSAMP-PROTO for consistency.
> >>
> >> -    Upper and lower CI limits can be also specified as provided
> >> absolute or relative limits. So we could also add 2 more IEs (for
> the
> >> relative CI limits)
> >>
> >>
> >>
> >> New description of IEs:
> >>
> >>
> >>
> >> absoluteError
> >>
> >> This Information Element specifies the maximum possible measurement
> >> error of the reported value for a given Information Element. The
> >> absoluteError has the same unit as the information
> > element
> >> it is associated to. The real value of the metric can differ by
> >> absoluteError (positive or negative) from the measured value. This
> >> information element provides only the error for measured values. If
> >> an information element contains an estimated values (from sampling)
> >> the confidence boundaries and confidence level have to be provided
> > instead.
> >>
> >>
> >>
> >> relativeError
> >>
> >> This Information Element specifies the maximum possible measurement
> >> error of the reported value for a given Information
> > Element
> >> as percentage of the measured value. The real value of the metric
> can
> >> differ by relativeError percent (positive or negative) from the
> >> measured value. This information element provides only the error for
> >> measured values. If an information element contains an estimated
> > values
> >> (from sampling) the confidence boundaries and confidence level have
> >> to be provided instead.
> >>
> >> I like your suggestions for absoluteError and relativeError because
> >> something that was not clear (neither from PSAMP-PROTO or PSAMP-
> INFO)
> >> is that we wanted to quantify the accuracy of the measurement
> >> estimation, as opposed to the accuracy of the estimated value
> >>
> >>
> >>
> >>
> >>
> >>
> >> upperCILimit
> >>
> >> This Information Element specifies the upper limit of a confidence
> >> interval. It is used to provide an accuracy statement for
> > an
> >> estimated value. The confidence limits define the range in which the
> >> real value is assumed to be with a certain probability p. Confidence
> >> limits always need to be associated with a confidence level that
> >> defines this probability p. Please note that a confidence interval
> > only
> >> provides a probability that the real values lies within the limits.
> >> That means the real value can lie outside the confidence limits.
> >>
> >>
> >>
> >> lowerCILimit
> >>
> >> This Information Element specifies the lower limit of a confidence
> >> interval. For further information see the description of
> >> upperCILimit.
> >>
> >>
> >>
> >> confidenceLevel
> >>
> >> This Information Element specifies the confidence level. It is used
> >> to provide an accuracy statement for estimated values. The
> confidence
> >> level provides the probability p with which the real value lies
> >> within a given range. A confidence level always needs to be
> >> associated with confidence limits that define the range in which the
> >> real value is assumed to be.
> >>
> >>
> >>
> >>
> >>
> >> Changes to PSAMP-PROTO if still possible:
> >>
> >>
> >>
> >> -    Rename fixedError to absoluteError
> >>
> >> -    Slightly modify paragraph 2
> >>
> >> OLD:
> >>
> >> ...  The accuracy SHOULD be reported either with the fixedError
> >> Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-
> >> psamp-protocol-09#ref-PSAMP-INFO> ], or with the relativeError
> >> Information Element [PSAMP-INFO
> > <http://tools.ietf.org/html/draft-ietf-
> >> psamp-protocol-09#ref-PSAMP-INFO> ].
> >>
> >> NEW:
> >>
> >> ... The accuracy for a measured information elelment SHOULD be
> >> reported either with the fixedError Information Element [PSAMP-INFO
> >> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> >> INFO> ], or with the relativeError Information Element [PSAMP-INFO
> >> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> >> INFO> ]. T
> >>
> >> To be consistent with my statement above, I would not add the
> > following
> >> sentence.
> >>
> >>
> >> he accuracy for an estimated information element (from sampling)
> >> SHOULD be reported with confidence limits and confidence
> >> level.[PSAMP- INFO]
> >>
> >>
> >>
> >> -    Remove the following paragraph (very important! Otherwise
> > it
> >> would lead to confusion):
> >>
> >> For example, the accuracy of an Information Element to estimate the
> >> accuracy of a sampled flow, for which the unit would be specified in
> >> octets, can be specified with the relativeError Information Element
> >> with the octet units.  In this case, the error interval is the
> >> Information Element value +/- the value reported in the
> relativeError
> >> times the reported Information Element value.
> >>
> >>
> >>
> >> -    Avoid the term error interval
> >>
> >> OLD:
> >>
> >> In this case, the error interval is the Information Element
> > value
> >> +/- the value reported in the fixedError.
> >>
> >> NEW:
> >>
> >> In this case, the real values lies within the range of the
> >> Information Element value +/- the value reported in the
> absoluteError.
> >>
> >>
> >>
> >>
> >>
> >> -    Remove the following paragraph (since absolute or relative
> >> error are just different representations I would not gain something
> >> if I report both)
> >>
> >> Alternatively to reporting either the fixedError Information Element
> >> or the relativeError Information Element in the Accuracy
> > Report
> >> Interpretation, both Information Elements MAY be present.  This
> >> scenario could help in more complex situations where the system
> clock
> >> drifts, on the top of having its own accuracy, during the duration
> of
> > a
> >> measurement.
> >>
> >> I would also change "Accuracy Report Interpretation" into
> >> "Measurement Accuracy Report Interpretation" in  [PSAMP-PROTO]
> >>
> >> Regards, Benoit.
> >>
> >>
> >>
> >>
> >>
> >>
> >> Sorry for the late comments, I was quite busy with PSAMP-TECH
> >> before...
> >>
> >>
> >>
> >> Kind regards,
> >>
> >> Tanja
> >>
> >>
> >>
> >>
> >>
> >

_______________________________________________
PSAMP mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/psamp