Re: [IPFIX] Export of long lived flow information
Gerhard Muenz <muenz@net.in.tum.de> Thu, 25 October 2012 18:26 UTC
Return-Path: <muenz@net.in.tum.de>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4ECEC21F8969 for <ipfix@ietfa.amsl.com>; Thu, 25 Oct 2012 11:26:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.949
X-Spam-Level:
X-Spam-Status: No, score=-5.949 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, HELO_EQ_DE=0.35, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 66fTgEmpHJ28 for <ipfix@ietfa.amsl.com>; Thu, 25 Oct 2012 11:26:29 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 86BB121F88A7 for <ipfix@ietf.org>; Thu, 25 Oct 2012 11:26:29 -0700 (PDT)
Received: from [192.168.2.36] (g229136115.adsl.alicedsl.de [92.229.136.115]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 3A9A419110DB; Thu, 25 Oct 2012 20:26:25 +0200 (CEST)
Message-ID: <50898454.2000706@net.in.tum.de>
Date: Thu, 25 Oct 2012 20:26:28 +0200
From: Gerhard Muenz <muenz@net.in.tum.de>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20121010 Thunderbird/16.0.1
MIME-Version: 1.0
To: John Court <johnwcrt@au1.ibm.com>
References: <OF96D061AA.F7F6CDD4-ONCA257AA0.00772818-4A257AA0.0078DF60@au1.ibm.com> <D50FAC55-C109-4A96-A471-538F27F9C2D9@tik.ee.ethz.ch> <OF30095AE1.689CF5C8-ONCA257AA1.001FB2C7-4A257AA1.00211D2B@au1.ibm.com> <5087B96B.7020500@cisco.com> <OFE375B6D9.49AD261E-ONCA257AA1.00703303-4A257AA1.00708F09@au1.ibm.com> <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com> <DE1ABD89-26A9-485E-893A-3160C6F671A6@cisco.com> <5088666F.1090106@cisco.com> <OF4B5A9A3A.F88C734E-ONCA257AA2.0005120F-4A257AA2.0005F365@au1.ibm.com>
In-Reply-To: <OF4B5A9A3A.F88C734E-ONCA257AA2.0005120F-4A257AA2.0005F365@au1.ibm.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 25 Oct 2012 18:26:31 -0000
Hi, It seems that some Information Element descriptions need clarification. For example, I think that the description of the Information Elements flowStart* and flowEnd* should be clarified: OLD: The absolute timestamp of the first|last packet of this Flow. NEW: The absolute timestamp of the first|last packet accounted in this Flow Record. I assume that this is how it is implemented in most existing implementations. Also, I assume that this is the intended meaning. My understanding is that both, deltaCounts and totalCounts contain the number of packets or octets observed in the indicated time interval. So, for identical flowStart* and flowEnd* timestamps, the values are the same. However, the description of totalCounts says that you report the number of packets or octets observed for this Flow since re-initialization. So, you must never reset the counter for this Flow, even after observing a FIN or RST. If you reset flow counters, or if you remove Flows from your Cache, you cannot use totalCounts any more unless you re-initialize the Metering Process (e.g. after flushing the entire permanent Cache). Using totalCounts, the flowStart* timestamp is identical in all Flow Records of the same Flow. Also, the collector knows that - for all but the first Flow Record of a Flow - the totalCount values include packets which were already reported in earlier Flow Records for the same Flow. Hence, each new Flow Record of the same Flow is an update of the previous ones. Summing up totalCount values in these Flow Records results in duplicate counts. On the other hand, with deltaCounts, the Flow Records refer to distinct time intervals. So, you can sum up counters without having duplicates. Although these subtle differences are not very obvious, the Information Element descriptions are quite clear. flowEndReason can be used to report some extra information but is not needed to understand the meaning of the Flow Records. Thanks, Gerhard On 25.10.2012 03:04, John Court wrote: > Yep I think everyone is starting to see the ambiguity that needs to be > cleared up :-) > > Paul, > > *Definitely. If it's a permanent flow and you're exporting totalCount > fields - which are measured "since the Metering Process > (re-)initialization for this Observation Point" - then the flowStartTime > must surely be the time the first ever packet was observed.* > > If you take that literally shouldn't that be interpreted to mean that > the totalCount continues into the next time a connection is up between > the same flow key ? Even if a flowEndReason of : > > 0x03: end of Flow detected > The Flow was terminated because the Metering Process > detected signals indicating the end of the Flow, for > example, the TCP FIN flag. > > That clearly wouldn't be of much use IMO and makes it difficult to see > what the flowEndReason field semantics mean in that context. Just > pointing out that taking that definition literally doesn't give a useful > answer on its own either :-). Although maybe that does make sense in a > router context ? Can you clarify this some more, perhaps you never > intend using the flowEndReason IE in your case ? > > Thanks > > > > > > From: Paul Aitken <paitken@cisco.com> > To: Andrew Johnson <andrjohn@cisco.com>, > Cc: Gerhard Muenz <muenz@net.in.tum.de>, John Court/Australia/IBM@IBMAU, > ipfix@ietf.org > Date: 25/10/2012 08:07 > Subject: Re: [IPFIX] Export of long lived flow information > ------------------------------------------------------------------------ > > > > Andrew, > > > I was thinking that a mechanism that allowed a non-permanent flow to > be exported multiple time would be useful. For example, security > applications generally want to know about a new flow ASAP, so they can > act on the information, but a short active timeout values lead to using > more export bandwidth. I was thinking we could do something like export > a report of the flow after the first packet, and then export the final > version of the flow once the normal timeouts had decided it was over. > > I have in the past discussed the idea of exporting a "new flow alert" > using zero-valued counters in order to make the collector aware that > we've started monitoring it - so I'm claiming prior art on that. > > > > I had in mind something like using a delta count, followed by a total > count. Reading the below definition of Total counts though, I'm not > sure that will work, but I think it depends on how we interpret the > definition of "Flow". If two records have matching key fields but > different starting timestamps, are they the same Flow? > > 5101 defines: > > A Flow is defined as a set of IP packets passing an Observation > Point in the network during a certain time interval. > > > - so it's all about the timestamps :-) > > > > I would argue that a single Flow can't have two flowStartTimes, so > maybe not. > > However, two flows with different flowStartTimes can be merged into one > flow. > > > > This would mean that we shouldn't reset the flowStartTimes between > sending reports for the same permanent Flow. > > Definitely. If it's a permanent flow and you're exporting totalCount > fields - which are measured "since the Metering Process > (re-)initialization for this Observation Point" - then the flowStartTime > must surely be the time the first ever packet was observed. > > P. > >
- [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information Andrew Johnson
- Re: [IPFIX] Export of long lived flow information Andrew Feren
- Re: [IPFIX] Export of long lived flow information Andrew Johnson
- Re: [IPFIX] Export of long lived flow information Tarun Saxena (tasaxena)
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Andrew Feren
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Gerhard Muenz
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Andrew Johnson
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Gerhard Muenz
- Re: [IPFIX] Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Andrew Feren
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information Andrew Johnson
- Re: [IPFIX] Export of long lived flow information Andrew Feren
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] [QUAR] Re: Export of long lived flow … Andrew Feren
- Re: [IPFIX] Export of long lived flow information Gerhard Muenz
- Re: [IPFIX] Export of long lived flow information Paul Aitken
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information Gerhard Muenz
- Re: [IPFIX] Export of long lived flow information Brian Trammell
- Re: [IPFIX] Export of long lived flow information John Court
- [IPFIX] Fw: Export of long lived flow information John Court
- Re: [IPFIX] Export of long lived flow information Brian Trammell