Re: [IPFIX] Export of long lived flow information

Paul Aitken <paitken@cisco.com> Wed, 24 October 2012 22:06 UTC

Return-Path: <paitken@cisco.com>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01ADB1F0C49 for <ipfix@ietfa.amsl.com>; Wed, 24 Oct 2012 15:06:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.532
X-Spam-Level:
X-Spam-Status: No, score=-10.532 tagged_above=-999 required=5 tests=[AWL=0.067, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4i6-tkC2gClb for <ipfix@ietfa.amsl.com>; Wed, 24 Oct 2012 15:06:43 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id DAAD21F0425 for <ipfix@ietf.org>; Wed, 24 Oct 2012 15:06:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1809; q=dns/txt; s=iport; t=1351116403; x=1352326003; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=p7E7h8dVeAsaXx6Oi20TVHwrPfgWhZYF7zsBv69z5kw=; b=kTzZNnraCL67skk8hEDgSEb3K9wd5ZWyRLVYwUTUWJlGw+9cQoX8PeDH T0wI90+Y0azyXf+gIEN2F+gDR5cPZbOl/MhqPk3nuLvoMFq2HaGTyzc71 mzne+Rj6doAuqECdD4q7XaWytOee1bSLOMPOxxcKBB+pIMKbipxp7U0Aj Q=;
X-IronPort-AV: E=Sophos;i="4.80,642,1344211200"; d="scan'208";a="145769132"
Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-1.cisco.com with ESMTP; 24 Oct 2012 22:06:42 +0000
Received: from [10.55.93.58] (dhcp-10-55-93-58.cisco.com [10.55.93.58]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9OM6fWk018357; Wed, 24 Oct 2012 22:06:41 GMT
Message-ID: <5088666F.1090106@cisco.com>
Date: Wed, 24 Oct 2012 23:06:39 +0100
From: Paul Aitken <paitken@cisco.com>
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Andrew Johnson <andrjohn@cisco.com>
References: <OF96D061AA.F7F6CDD4-ONCA257AA0.00772818-4A257AA0.0078DF60@au1.ibm.com> <D50FAC55-C109-4A96-A471-538F27F9C2D9@tik.ee.ethz.ch> <OF30095AE1.689CF5C8-ONCA257AA1.001FB2C7-4A257AA1.00211D2B@au1.ibm.com> <5087B96B.7020500@cisco.com> <OFE375B6D9.49AD261E-ONCA257AA1.00703303-4A257AA1.00708F09@au1.ibm.com> <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com> <DE1ABD89-26A9-485E-893A-3160C6F671A6@cisco.com>
In-Reply-To: <DE1ABD89-26A9-485E-893A-3160C6F671A6@cisco.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Cc: John Court <johnwcrt@au1.ibm.com>, ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 24 Oct 2012 22:06:44 -0000

Andrew,

> I was thinking that a mechanism that allowed a non-permanent flow to be exported multiple time would be useful.  For example, security applications generally want to know about a new flow ASAP, so they can act on the information, but a short active timeout values lead to using more export bandwidth.  I was thinking we could do something like export a report of the flow after the first packet, and then export the final version of the flow once the normal timeouts had decided it was over.

I have in the past discussed the idea of exporting a "new flow alert" 
using zero-valued counters in order to make the collector aware that 
we've started monitoring it - so I'm claiming prior art on that.


> I had in mind something like using a delta count, followed by a total count.  Reading the below definition of Total counts though, I'm not sure that will work, but I think it depends on how we interpret the definition of "Flow".  If two records have matching key fields but different starting timestamps, are they the same Flow?

5101 defines:

       A Flow is defined as a set of IP packets passing an Observation
       Point in the network during a certain time interval.


- so it's all about the timestamps :-)


> I would argue that a single Flow can't have two flowStartTimes, so maybe not.

However, two flows with different flowStartTimes can be merged into one 
flow.


> This would mean that we shouldn't reset the flowStartTimes between sending reports for the same permanent Flow.

Definitely. If it's a permanent flow and you're exporting totalCount 
fields - which are measured "since the Metering Process 
(re-)initialization for this Observation Point" - then the flowStartTime 
must surely be the time the first ever packet was observed.

P.