Re: [IPFIX] unobserved fields

[Paul Aitken <[e-mail]>]

Brian,

> Ahhhh -- okay, so you'd be proposing using an indexing mechanism, as with MIB objects, to explicitly link one IE in a record with another to allow the "observation index" IE to expose the observation state of each IE?

No. There's only one IE.

In MIB export, the template effectively says "here is a MIB, with two 
extensions; the first defines the OID (in the template); the second 
defines the index (in the data records)". Appropriate sizes are given, 
and the template and data records are extended accordingly. So instead 
of getting a 4-byte value in the data record, there might be 6 bytes: 
the 4 bytes value followed by a 2-bytes index. However the index has no 
IE of its own, so each new extension does not require a new IE.

In the unobserved fields scenario, the template would says "here is an 
IE, with an extension (in the data records)", and that extension would 
indicate the observability of the IE. So instead of 4 bytes of RTT time, 
there might be 5 bytes: 4 bytes of RTT (being zero, presumably), and 1 
byte reporting "insufficient time".


> I think an example illustrating this -- how MIB-style EFS would support -unobserved-fields -- in the next rev of -unobserved-fields would be very useful.

Are MIBS ever unobserved?


> A couple of other points inline...
>
> On 9 Dec 2012, at 19:01, Paul Aitken <[e-mail]> wrote:
>
>>> As I understand the requirements for -unobserved-fields, we want a way to distinguish, from an MP: "I am capable of observing this attribute, but it was not present in the observation"
>> Or, I don't (yet) have sufficient data to calculate a derived metric, eg average / min / max, count or time based metrics.
>>
>>
>>> versus "I am not capable of observing this attribute". We have the second already: simply leave the IE out of the template.
>> No. Leaving the metric out of the template does not actively indicate that the MP is not capable of measuring it. A CP cannot make any assumptions about metrics which the MP isn't reporting. Rather than this *passive* method, we're missing a way for the MP to *actively* inform the CP that although it's capable of measuring some metric, it's currently not able to provide any further information about the metric.
> Absolutely agree; I'm pointing out the "way to do this today" -- which is ambiguous.

Great, we agree.


>>> And, indeed, you can also use leaving things out of the template to signify the first: the presence of a field in some templates from an EP/MP combination implies that it's observable, its absence in the other implies it's not observed.
>> No it doesn't. That's the whole point of -unobserved-fields! The MP could have observed it and between the MP and EP, decided not to export it. eg, certain IP addresses, user names, URLs, could be filtered from export stream for privacy or security. The absence of a field in the template isn't directly related to the field's observability.
> As above. I'm not saying this is the right way to do things; I'm saying it's the de facto approach.

Which has this huge gap...


>>> FWIW, YAF does this for those things that haven't moved behind the 6313 curtain: no reverse elements if it's not a biflow, for instance.
>>>
>>> Given that, how would you use a per-IE specifier to signal a per-record attribute? What am I missing?
>> EFS are also per-record, eg OID index. See the MIB draft.
> Okay, so EFS -- specificially, any extension applying an "indexing" feature -- allows specifying that one IE in a Template is present to modify another, so the per-record information still appears in the record; it's just that EFS has allowed an explicit linkage between the IEs, where before we'd have only an implicit linkage (bad, because it's very easy to interpret incorrectly) or an explicit linkage keyed to a specific function in the definition of the IE, e.g. flowKeyIndicator (less bad but still bad, because continuing along these lines you run out of IEs pretty quickly)...correct?

Sort of, except there's only one IE with additional data. I'm not 
proposing new IEs for every new attribute specified in an EFS.

P.