[IPFIX] Draft Standard version of RFC5101

[Benoit Claise <[e-mail]>]

Dear all,

Here is the list of things to be changed in the Draft Standard version 
of RFC5101.
Along the years, I've been collecting (from different people a list of 
what could be improved in IPFIX. Many points come from discussions with 
Paul Aitken.

Some of points below should be discussed.

1. what must be changed is obviously 
http://www.rfc-editor.org/errata_search.php?rfc=5101

2. the email thread 
http://www.ietf.org/mail-archive/web/ipfix/current/msg05652.html 
suggests to change the definition from "IP Traffic Flow or Flow" to 
"Traffic Flow or Flow", and from "IP packets" to "packets" when RFC5101 
will go from Proposed Standard to Draft Standard.

3. Different Template Record management between the transport protocol 
(feedback from the IPFIX interop)

SCTP transport

    If the Collecting Process receives a Template that has
    already been received but that has not previously been withdrawn
    (i.e., a Template Record from the same Exporter Observation Domain
    with the same Template ID received on the SCTP association), then the
    Collecting Process MUST shut down the association.


So an identical Template Record is resent (like in UDP), the Collecting 
Process MUST shut down the association
First problem: we don't have the same statement for TCP, which only says:

    If the Collecting Process receives a malformed IPFIX Message, it MUST
    discard the IPFIX Message and SHOULD log the error.

So this is not consistent as we don't know what "malformed" means.
Thinking some more about it, is this really a mistake if SCTP and TCP 
would behave as UDP, i.e. accept the exact same (Options) Template 
Record to be resent? Proposal with something such as: It's NOT 
RECOMMENDED that _identical _Template Record are resent in SCTP and 
TCP.... but it will not cause the SCTP association or TCP association to 
be shut down...
Before applying this change, we have to double-check if there are no 
complications for draft-ietf-ipfix-export-per-sctp-stream-08

4.

OLD:

Exporting Process ID
                         The identifier of the Exporting Process for
                         which lack of reliability is reported.  There
                         are three Information Elements specified in
                         [RFC5102  <http://tools.ietf.org/html/rfc5102>] that can be used for this purpose:
                         exporterIPv4Address, exporterIPv6Address, or
                         exportingProcessId.  This Information Element
                         MUST be defined as a Scope Field.

NEW:

Exporting Process ID
                         The identifier of the Exporting Process for
                         which lack of reliability is reported.  There
                         are three Information Elements specified in
                         [RFC5102  <http://tools.ietf.org/html/rfc5102>] that can be used for this purpose:
                         exporterIPv4Address, exporterIPv6Address, or
                         exportingProcessId.  This Information Element,
                         _or these Information Elements if multiple are
_                         _needed,_  MUST be defined as a Scope Field

Justification: both the address and process ID are needed to identify a 
specific exporter in the case  of multiple exporters at the same address.



5.

10.3.6.  Template Management

    Following a configuration change that can modify the interpretation
    of the Data Records (for example, a sampling rate change) a new
    Template ID MUST be used, and the old Template ID MUST NOT be reused
    until its lifetime (see Section 10.3.7) has expired.

In section 10.3.7.  Collecting Process, we read:

    The Template lifetime at the Collecting Process MUST be at least 3
    times higher than the Template refresh timeout configured on the
    Exporting Process.


In Flexible NetFlow, template refresh is in units of packets, so that 
template export scales with the volume of exported data.
If we're no longer exporting any packets then the "3 times higher" 
threshold will never be reached... so the templates will never expire.
eg, if we're exporting templates every 1,000 packets, then the lifetime 
on the collector should be 3,000 packets. Yet once we're no longer using 
the template, we'll be exporting 0 packets...
So the RFC should specify the units, or give an alternative mechanism.


6.

Section 8:
    If the measurement parameters change such that a new Template is
    required, the Template MUST be withdrawn (using a Template Withdraw
    Message and a new Template definition) or an unused Template ID MUST
    be used.  Examples of the measurement changes are: a new sampling
    rate, a new Flow expiration process, a new filtering definition, etc.

A new sampling rate etc may not require a new template at all; it may be 
expressed with the same template, just different data. The template 
belongs to the transport layer and really has nothing to do with the 
sampler change!

Proposal: remove "a new sampling rate"


7.

Section 4.3.  The Exporting Process Reliability Statistics Option Template

    time first flow dropped
                         The timestamp of the first Flow was dropped by
                         the Metering Process.  For this timestamp, any
                         of the "flowStart" timestamp Information
                         Elements flowStartMilliseconds,
                         flowStartMicroseconds, flowStartNanoseconds, and
                         flowStartDeltaMicroseconds can be used.

    time last flow dropped
                         The timestamp of the last IP packet that was
                         ignored by the Metering Process.  For this
                         timestamp, any of the "flowEnd" timestamp
                         Information Elements flowEndMilliseconds,
                         flowEndMicroseconds, flowEndNanoseconds, and
                         flowEndDeltaMicroseconds can be used.

Firstly, these definitions are inconsistent since the names and the 
first definition say "flow" while the second definition says "IP 
packet". Obviously "IP packet" != "flow" :-o

Secondly, "The timestamp of the first Flow was dropped by the Metering 
Process." is bad English: at least it's missing "that".

8.

Section 4.3.  The Exporting Process Reliability Statistics Option Template

These statistics where defined before we introduced the concept of 
Transport Session.
  This full section 3.4 should be re-specified for "4.3 The Transport 
Session Reliability Statistics Option Template"


9.

Section 4.2.  The Metering Process Reliability Statistics Option Template

Suppose a packet cannot be classified before it's dropped. Which
Metering Process (meteringProcessId) should report this?

Suppose the packet pertains to several monitors. If some or all of
them cannot record the packet (eg, cache full), should they all report
the loss? That might look like some random number of packets had been
lost, when in fact it's only one.



10.

Do we want to have a new IE  for the template lifetime, to be sent in an 
Options Template Record.


Regards, Benoit.