Re: [IPFIX] WG Last call for draft-ietf-ipfix-mediation-protocol-06.txt

[Brian Trammell <[e-mail]>]

Hi, Andrew,

Apologies for letting this one sail by unanswered, and many thanks for the review. Comments thereon inline.

On Aug 30, 2013, at 4:52 PM, Andrew Feren <[e-mail]> wrote:

> Hi Nevil, all,
> 
> I missed last call, but as no one else has posted anything here are my comments.
> 
> First, I think something like this document is needed, but I'd prefer the language to be a bit more general in a number of places.  I also have a number of specific issues.

This has been an extremely difficult document to write for exactly this reason -- we have tried to be as general as possible, but too general and the document says nothing, while too specific and it's unclear if the section is to have general applicability.

> An IPFIX Mediator converts a record stream to IPFIX and a record stream can be "IPFIX Data Records or any other format", but other than those points in the Terminology section this document seems to assume IPFIX input.
> 
> For example:
>   Original Exporter:   An Original Exporter is an IPFIX Device that
>      hosts the Observation Points where the metered IP packets are
>      observed.
> 
> I have a need for something very much like originalExporterIPv4Address except that my data source(s) fall under that "any other format" umbrella and are not IPFIX devices. I'd prefer to see originalSenderIPv4Address (or some such) rather than an IE for each non-IPFIX source.

I actually think that the "original exporter" concept can be extended to "the place I got this data from, regardless of how"; I'd be in favor of striking IPFIX Device from the definition. Note: there may be effects on already registered IEs here, will have to check.

> If there is a need to know that the original source was IPFIX or something else I'd prefer an additional IE for originalSenderFormat.  Probably with a registry of known formats. I don't have a use case for this IE at the moment, but perhaps there is a need.

Why would there be? 

For archival purposes, it might be useful to have metadata about the protocols/representations in order to make judgements based on design characteristics/design flaws therein; however, if you're _really_ getting into storing archival information, you need implementation details and patchlevels. A complete data provenance framework for IPFIX would be an interesting problem to tackle if we had a few spare years, but I think we're pretty firmly in diminishing returns territory here.

> In section 3 "Note that the format is compatible
>   with the IPFIX Message Header defined in
>   [I-D.ietf-ipfix-protocol-rfc5101bis], with some field definitions
>   (for the example, the Export Time) updated in the context of the
>   IPFIX Mediator."
> 
> I don't like this at all.  Does an exporter send IPFIX or something compatible with IPFIX?

The philosophical question: is there something compatible with IPFIX that is not IPFIX? Though in terms of wording, "interoperable with" may be better. Actually, there are a couple of problems with this paragraph on review. Your replacement text below is a definite improvement.

> For example Export Time "MAY use the export time received from the incoming Transport Session".  I think I understand the motivation behind this, but at the collector I have no way to know which export time is being used.  How do I know if the time applies to the original or mediator?  

Why do you care? The only use for Export Time is as a basetime for the delta timestamps (which themselves are  not recommended in cases where mediators apply, see Section 7 paragraph 2) and for template action sequencing (as in Section 8.2 of RFC 7011). I guess it could also be useful to display to the operator for debugging the infrastructure. Any other assumptions based on Export Time are not covered by the warranty. :)

> I'd rather not see this exception unless someone can explain how it is useful.
> 
> The text for sequence number doesn't match 5101bis.  More generally I think it is confusing to include all that copy paste text in this document.  I'd rather see a reference to 5101bis followed by a list of special circumstances.
> 
> Something like
>     The format of the IPFIX Message Header as exported by an IPFIX
>     Mediator is the same as  [RFC5102bis].
> 
>      See Section 4.1 for special considerations for Observation Domain
>      management while passing unmodified templates through an IPFIX
>      Mediator, and Section 5 for guidelines for preservation of
>      original Observation Domain information at an IPFIX Mediator.

Yep, this is good; will take it into a new rev.

> That makes it obvious what is special or different(*shudder*) relative to IPFIX.

Again, the changes here are not a problem from an interoperability standpoint. You shouldn't be relying on any assumptions about export time beyond the guarantees in 7011 section 8.2, or on _any_ information about the Observation Domain other than it's the root scope for options and template IDs.

> Section 4.1 and 4.3 discuss two situations when a Mediator MUST NOT reorder and fields in a record.  Additionally, if an IE occurs more than once in a template that order also MUST NOT be modified.  I would think that this situation should also be treated similarly to unknown IEs either resend all or none.
> 
> 4.2 starts with "The second case".  I'm not entirely sure what was first.
> 
> In section 4.3
> 
> " Depending on application requirements, Mediators which do not
>   generate new Records SHOULD re-export values for unknown Information
>   Elements, whether enterprise-specific Information Elements or
>   Information Elements in the IPFIX Information Element registry
>   [iana-ipfix-assignments]. added since the Mediator was implemented or
>   updated."
> 
> That sentence seems a bit long and complicated.  At the very least I think the '.' before added needs to be removed.  

Yep.

> Although I'd prefer to simplify it.  I think it is enough to state that "An IE is considered unknown if its data type and semantics are not known to the Mediator."  Trying to list the reasons it might not be known just complicates things IMO.

Also yep. Thanks.

> I didn't fully grok Section 5, but "[RFC6313] MUST be used" bothers me.

The intention here is MAY export, if export MUST 6313. The language should be clearer about that.

> Mostly because from a collector perspective I really don't like 6313 at all and subTemplateMultiList in particular.  Is 6313 really the only way to export this data?  Why?

Well, you could have a template with multiple instances of an OP identifier, or multiple records scoped with options, neither of which is pretty. Whether I think it's more or less pretty than 6313 I'll leave as an exercise to the reader. The problem isn't the complexity of 6313, though, it's the complexity of the underlying measurement surface. Encoding it differently won't make that complexity go away.

On the other hand, it's unclear that given arbitrarily complex semantics that a collector can or should do anything with this _other_ than simply stick it on disk somewhere for subsequent human interpretation.

> Section 7 timing constraints refers to relative timestamps as "difficult to manage".  I think "impossible to manage" is closer to the mark.  A collector can get sysUpTime from a v9 header.  For IPFIX I'm no sure where I get sysUpTime from.  Perhaps there needs to be a requirement for Mediators converting v9 to IPFIX to translate relative times to absolute times.

Most of the time, yes. There are simple cases, and again, we're trying to be as general as we can while still having something to say.

Will spin out a -07 revision in the next couple of days with the edits here. Again, many thanks for the review!

Cheers,

Brian



> Better late than never I guess,
> -Andrew
> 
> On 08/26/2013 11:32 PM, Nevil Brownlee wrote:
>> 
>> Hi all:
>> 
>> This WGLC has finished, with no comments at all on the list.
>> I need at least a few "looks OK to me" or "I can live with that"
>> responses before I can put together its Shepherd writeup!
>> 
>> Cheers, Nevil
>> 
>> 
>> On 8/08/13 10:12 AM, Nevil Brownlee wrote:
>>> 
>>> Hi all:
>>> 
>>> The -06 version of IPFIX Mediation Protocol was published last week,
>>> now it's time for its WG Last Call.
>>> 
>>> The WGLC starts now, and will run until Friday, 23 August.
>>> 
>>> Please read the draft and send your comments/suggestions to the list.
>>> Actual reviews of the draft would be much appreciated, but a brief
>>> comment saying "I read it, it's fine" is useful too!
>>> 
>>> Cheers, Nevil
>> 
> 
> _______________________________________________
> IPFIX mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ipfix