Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?

"Heinbockel, Bill" <> Thu, 17 February 2011 16:36 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 17B5D3A6C7D for <>; Thu, 17 Feb 2011 08:36:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id ClNBL3UcbgSy for <>; Thu, 17 Feb 2011 08:36:01 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id AC3AD3A6C3C for <>; Thu, 17 Feb 2011 08:36:01 -0800 (PST)
Received: from (localhost.localdomain []) by localhost (Postfix) with SMTP id A1DBC21B17F3; Thu, 17 Feb 2011 11:36:32 -0500 (EST)
Received: from imchub2.MITRE.ORG ( []) by (Postfix) with ESMTP id 8F71321B17D3; Thu, 17 Feb 2011 11:36:32 -0500 (EST)
Received: from IMCMBX3.MITRE.ORG ([]) by imchub2.MITRE.ORG ([]) with mapi; Thu, 17 Feb 2011 11:36:32 -0500
From: "Heinbockel, Bill" <>
To: Jeroen Massar <>
Date: Thu, 17 Feb 2011 11:35:19 -0500
Thread-Topic: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
Thread-Index: AcvNxSVcMeG8rWBOQCCWiPHcleUWdwA+p2yQ
Message-ID: <93ED0A84F9A1D74FA65021D940AA5884054477F158@IMCMBX3.MITRE.ORG>
References: <> <93ED0A84F9A1D74FA65021D940AA588405446C41F9@IMCMBX3.MITRE.ORG> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
acceptlanguage: en-US
Content-Type: multipart/signed; micalg=SHA1; protocol="application/x-pkcs7-signature"; boundary="----=_NextPart_000_0033_01CBCE96.C12A9240"
MIME-Version: 1.0
Cc: Sam Johnston <>, Event Expression <>, "" <>, Common
Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Security Issues in Network Event Logging <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 17 Feb 2011 16:36:04 -0000

Thanks for that, I did not realize that IPFIX had been extended beyond its netflow 

I don't have the time now, but I am interested in looking into it further. It does 
kind of remind me of ASN.1/SNMP, where we need to worry about the names/OID 

Also, a lot of vendors and users seem to prefer the ease of text-based protocols 
like Syslog for logging. I am not saying this is good or bad, but it seems to be 
the sweetspot -- binary is not natively human readable and XML has too much 

William Heinbockel
The MITRE Corporation

>-----Original Message-----
>From: Jeroen Massar []
>Sent: Wednesday, 16 February 2011 05:35
>To: Heinbockel, Bill
>Cc:; Chris Lonvick; Gene Golovinsky; Sam Johnston; Common
>Event Expression
>Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
>On 2011-02-16 06:21, Heinbockel, Bill wrote:
>> From what I understand, IPFIX is for expression of IP flows from network
>> devices.
>For a short bit forget about the history of IPFIX, it indeed comes from
>NetFlow, and thus is used quite in a network centric way, but
>effectively it is a structured streaming data format.
>> Could you please explain how IPFIX is relevant to event and cloud logging
>> I understand how CEE and IPFIX may overlap for describing networking
>events, but
>> it is unclear to me how IPFIX could handle things like Windows Event Logs
>> RHEL audit logs.
>There are two parts to IPFIX: Templates + Data
>The template describes how the data looks like, for instance, lets take
>an Apache CLF log entry:
> - - [16/Feb/2011:10:48:11 +0100] "GET /robots.txt
>HTTP/1.1" 200 2629 "-" "Googlebot-Image/0"
>We can make an IPFIX template for that
>	{4, IPv4_SRC },
>	{v, URL},
>	{8, OCTETS},
>	{v, HTTP_REFER},
>The 'v' markers indicate variable fieldlengths, the others indicates the
>number of bytes such a field takes. The data is then just encoded in the
>above format, presto.
>The above is a simple example, one can also have repeating lists and of
>course you could make a variable template which just includes the fields
>that you actually want to look at or you could already do some
>aggregation and add other fields. Templates are only sent every now and
>then, as they should not change. The data is the important bit.
>The fieldnames are actually numbers in the data, thus very compact, and
>are mapped to descriptions, data types etc, per a nice XML file
> (or .xhtml or .txt for
>a more human readable version ;) for the official IANA list and with the
>help of Enterprise IDs any others can easily be added.
>The big advantage is that you can more or less do static templates if
>you want and you only need one single parser on the collector side, thus
>one does not have to create another parser and collector again for
>decoding other protocols, just one, the IPFIX one, and you can optimize
>that really well for all kinds of scenarios.
> Jeroen