RE: [ipfix] drawing the line between protocol and data model docu ment

["MEYER,JEFFREY D (HP-Cupertino,ex1)" <[e-mail]>]

Benoit,
 
  From a purely practical matter.  What does NFv9 do today?  Are any
counters 64-bit (e.g. length=8) or are they all 32-bit (e.g. length=4).  Do
you anticipate doing things like length=7, length=33?
 
  If you are saying that this kind of "flexibility" is a good thing.  I.e. I
should write a consuming app to accept a template which might have
length="11" vs. knowing that the data model explicitly is sending me values
of some more constrained type (e.g. an integer in the range of 0-2^32), I
disagree.
 
  Again, I don't see a need to change the encoding, continue sending length.
But for items defined in the information model, be EXPLICIT about their
capacity.  Don't just say "its numeric".  
 
  If we need 128-bit counters later, fine, we can extend the information
model, and give specific names to these items and the protocol will continue
to work w/o change.
 
Regards,
 
  Jeff Meyer
 
 -----Original Message-----
From: Benoit Claise [mailto:[e-mail]]
Sent: Thursday, June 12, 2003 5:47 AM
To: Tal Givoly
Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); Mark Fullmer;
[e-mail]; [e-mail]
Subject: Re: [ipfix] drawing the line between protocol and data model
document



Tal, Jeff,

[let me reply to both of you in a single email]


Benoit,
 
I agree with Jeff. I would add that, no more than 2 options should be
available for bytes/packet counters (32/64-bit unsigned integers).
 
But I would consider offering only a 64-bit value for byte counters. From
SNMP we learned that 32-bit byte counters cause countless problems. Since
IPFIX byte counters may be a result of aggregates, with today's networks it
is very feasible to overflow the 32-bit integer container.

How often do you poll the SNMP variables like interface counters? 5 minutes?
This is order of magnitude higher than the flow expiration, in most of the
cases.

http://www.ietf.org/internet-drafts/draft-claise-netflow-9-02.txt
<http://www.ietf.org/internet-drafts/draft-claise-netflow-9-02.txt> 
    -> 3.2 Flow Expiration

   A Flow is considered to be inactive if no packets belonging to the 

   Flow have been observed at the Observation Point for a given timeout 

   interval otherwise it is considered as an active flow.  

   A Flow can be exported under the following conditions: 

    

      1. If the Exporter can detect the end of a Flow, it SHOULD export 

      the Flow Records at the end of the Flow. For example, a Flow 

      generated by TCP [3] type of traffic where the FIN or RST bits 

      indicate the end of the Flow. 

       

      2. If the Flow has been inactive for a certain period of time. 

      This inactivity timeout SHOULD be configurable, with a minimum 

      value of 0 for an immediate expiration. For example, a Flow 

      generated by UDP [2] type of traffic. 

       

      3. For long-lasting Flows, the Exporter SHOULD export the Flow 

      Records on a regular basis. This periodicity SHOULD be 

      configurable. 

       

      4. If the Exporter experiences internal constraints, a Flow MAY 

      be forced to expire prematurely (for example, counters wrapping 

      or low memory). 



So it's only in case 3 (so potentially an aggregation, as you described)
that we could potentially needs a higher counter.

And in that case again, a simple solution is to export the flow records just
before it reaches the max value of counter32 (in case the length for
counter32 is choosen).



But if you don't want that, there is also another solution: when this case
3. occurs, you can define a new template with the length corresponding to
counter64.

And then you have 2 templates: one with counter32 (used most of the time)
and one with counter64 (used in a few cases).



Another solution, nothing prevents you to export flow data records always
with counter64



See below...

 
Tal

-----Original Message-----
From: majordomo listserver [ mailto:[e-mail]
<mailto:[e-mail]> ]On Behalf Of MEYER,JEFFREY D
(HP-Cupertino,ex1)
Sent: Wednesday, June 11, 2003 10:00 AM
To: 'Benoit Claise'
Cc: Mark Fullmer; [e-mail] <mailto:[e-mail]>
; [e-mail] <mailto:[e-mail]> 
Subject: RE: [ipfix] drawing the line between protocol and data model
document


Hi,
 
  >From a practical perspective, simply saying that a field is numeric and
may be encoded as 1 or 100 bytes seems like it puts an unnecessary and
unjustified burden on the collector, or any other app which might want to
handle this data.

Am I correct that the burden would only be when you decode the template
records?
Because when you receive a Data FlowSet, the collecting process must read
the FlowSet ID and automatically know the data type length.


 
  Is there some particular reason why saying a specific information item,
e.g. numBytes, is a 32-bit integer or a 64-bit integer is considered
undesireable. 


But then, why to limit to 32-bit and 64-bit integer? What about other
possibilities? Where to stop? Then we end up with a long information model
because this principle of 32-bit, 64-bit integer should be then applied to
all data type that are counters.


It certainly seems like the developer of a consuming application would
benefit from knowing how to target variables in their application for given
information items.  I guess one could assume that everything will fit in a
64-bit integer and code your app that way, but again, this seems like an
unnecessary burden for unclear gains.

The gain, as I see it, is in the flexibility with a simple information model

Regards, Benoit.



 
  I don't think anything needs to change in the protocol.  Simply a more
constrained approach to dealing with numeric fields with explicit types.
 
Regards,

  Jeff Meyer

-----Original Message-----
From: Benoit Claise [ mailto:[e-mail] <mailto:[e-mail]> ]
Sent: Wednesday, June 11, 2003 4:31 AM
To: Benoit Claise
Cc: Mark Fullmer; [e-mail] <mailto:[e-mail]>
; [e-mail] <mailto:[e-mail]> 
Subject: Re: [ipfix] drawing the line between protocol and data model
document


Dear all,


Mark, 

On Thu, May 01, 2003 at 10:51:49AM -0400,  [e-mail]
<mailto:[e-mail]>  wrote:



<snip>



  

The richness of the protocol encoding specification is another

matter. The protocol would reference each element in the data

model and define exaclty how that element is encoded. In the above 

example, the protocol is free to encode that as an unsigned byte 

if the value is < 256. 

    



The ramifications of doing this with a template based system are a bit

ugly.  Consider a flow with just 2 counters -- octets, packets.

Each could have a 1 to 8 byte encoding.  That's a potential of 64

different templates for representing 1 type of flow.



This (at first glance) looks like it adds considerable cycles to the

encode / decode process to save a few bytes in the transport.



The current v9 spec appears to allow this, but only for IN_BYTES and

IN_PKTS.



  

                                          counter with length

   IN_BYTES                 1       N     N x 8 bits for bytes

                                          associated with an IP Flow

    



Is this true?  Benoit?

  

I'm trying to catch up with my IPIFX emails; I know I'm late on this thread,
which seems to have reached consensus as far as I can tell, but as this is a
direct question to me, let me answer.
Actually, there are:
BYTES            1             4             field type for the 32 bit bytes
counter
BYTES_64      23           8             field type for the 64 bit bytes
counter

Let me reply to my own email because, after speaking to different people, I
have to correct what I said.
We actually do:

                                          Incoming counter with length

   IN_BYTES                 1       N     N x 8 bits for bytes

                                          associated with an IP Flow



                                          Incoming counter with length

   IN_PKTS		    2       N     N x 8 bits for packets

                                          associated with an IP Flow



And no BYTES_64 is defined.



Regards, Benoit.







So the draft that you refer to needs a quick update.
<!--[endif]-->
Regards, Benoit





mark



--

Help         mailto:[e-mail]
<mailto:[e-mail]>  and say "help " in message body
<inmessagebodyUnsubscribemailto:[e-mail]> 

Unsubscribe mailto:[e-mail] and say

"unsubscribe ipfix" in message body

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/>