From: Gene Golovinsky <gene@alertlogic.com>
To: Dan Schlitt <schlitt@world.std.com>, "Heinbockel,
 Bill" <heinbockel@mitre.org>, "clouds@ietf.org" <clouds@ietf.org>
Date: Thu, 17 Feb 2011 12:47:33 -0600
Thread-Topic: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
Thread-Index: AcvOxxvimbNTmqoKTxOIAS4W5JyZrQAC/PPQ
Message-ID: <C6A1D07CACFDBD4D9422C7D7ED288D4104877B57B9@34093-MBX-C01.mex07a.mlsrvr.com>
References: <4D5A60C8.3090000@unfix.org>
 <93ED0A84F9A1D74FA65021D940AA588405446C41F9@IMCMBX3.MITRE.ORG>
 <4D5BA85B.7040007@unfix.org>
 <93ED0A84F9A1D74FA65021D940AA5884054477F158@IMCMBX3.MITRE.ORG>
 <Pine.SGI.4.61.1102171213350.5801294@shell01.TheWorld.com>
In-Reply-To: <Pine.SGI.4.61.1102171213350.5801294@shell01.TheWorld.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Sam Johnston <sj@google.com>, Event Expression <cee@mitre.org>,
 "syslog@ietf.org" <syslog@ietf.org>,
 "Common@core3.amsl.com" <Common@core3.amsl.com>
Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
Precedence: list

Is there a link to the article?

--Gene



-----Original Message-----
From: syslog-bounces@ietf.org [mailto:syslog-bounces@ietf.org] On Behalf Of=
 Dan Schlitt
Sent: Thursday, February 17, 2011 11:21 AM
To: Heinbockel, Bill
Cc: Sam Johnston; Event Expression; syslog@ietf.org; Common@core3.amsl.com
Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?


On this topic it might be well to look at Tom Limoncelli's article in the F=
ebruary Communications of the ACM. He is highly respected in the system adm=
inistrator community and speaks for many of us.

/dan

--=20

Dan Schlitt
schlitt@world.std.com


On Thu, 17 Feb 2011, Heinbockel, Bill wrote:

> Thanks for that, I did not realize that IPFIX had been extended beyond=20
> its netflow past.
>
> I don't have the time now, but I am interested in looking into it=20
> further. It does kind of remind me of ASN.1/SNMP, where we need to=20
> worry about the names/OID translation
>
> Also, a lot of vendors and users seem to prefer the ease of text-based=20
> protocols like Syslog for logging. I am not saying this is good or=20
> bad, but it seems to be the sweetspot -- binary is not natively human=20
> readable and XML has too much overhead.
>
>
> William Heinbockel
> The MITRE Corporation
>
>
>> -----Original Message-----
>> From: Jeroen Massar [mailto:jeroen@unfix.org]
>> Sent: Wednesday, 16 February 2011 05:35
>> To: Heinbockel, Bill
>> Cc: syslog@ietf.org; Chris Lonvick; Gene Golovinsky; Sam Johnston;=20
>> Common Event Expression
>> Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
>>
>> On 2011-02-16 06:21, Heinbockel, Bill wrote:
>>> From what I understand, IPFIX is for expression of IP flows from=20
>>> network
>> sensing
>>> devices.
>>
>> For a short bit forget about the history of IPFIX, it indeed comes=20
>> from NetFlow, and thus is used quite in a network centric way, but=20
>> effectively it is a structured streaming data format.
>>
>>> Could you please explain how IPFIX is relevant to event and cloud=20
>>> logging
>> data?
>>> I understand how CEE and IPFIX may overlap for describing networking
>> events, but
>>> it is unclear to me how IPFIX could handle things like Windows Event=20
>>> Logs
>> and
>>> RHEL audit logs.
>>
>> There are two parts to IPFIX: Templates + Data
>>
>> The template describes how the data looks like, for instance, lets=20
>> take an Apache CLF log entry:
>>
>> 66.249.66.174 - - [16/Feb/2011:10:48:11 +0100] "GET /robots.txt=20
>> HTTP/1.1" 200 2629 "-" "Googlebot-Image/0"
>>
>> We can make an IPFIX template for that
>>
>> [
>> 	{4, IPv4_SRC },
>> 	{4, TIMESTAMP},
>> 	{4, HTTP_METHOD},
>> 	{v, URL},
>> 	{v, HTTP_PROTOCOL},
>> 	{2, HTTP_RESULT},
>> 	{8, OCTETS},
>> 	{v, HTTP_REFER},
>> 	{v, HTTP_USERAGENT},
>> ]
>>
>> The 'v' markers indicate variable fieldlengths, the others indicates=20
>> the number of bytes such a field takes. The data is then just encoded=20
>> in the above format, presto.
>>
>> The above is a simple example, one can also have repeating lists and=20
>> of course you could make a variable template which just includes the=20
>> fields that you actually want to look at or you could already do some=20
>> aggregation and add other fields. Templates are only sent every now=20
>> and then, as they should not change. The data is the important bit.
>>
>> The fieldnames are actually numbers in the data, thus very compact,=20
>> and are mapped to descriptions, data types etc, per a nice XML file=20
>> http://www.iana.org/assignments/ipfix/ipfix.xml (or .xhtml or .txt=20
>> for a more human readable version ;) for the official IANA list and=20
>> with the help of Enterprise IDs any others can easily be added.
>>
>> The big advantage is that you can more or less do static templates if=20
>> you want and you only need one single parser on the collector side,=20
>> thus one does not have to create another parser and collector again=20
>> for decoding other protocols, just one, the IPFIX one, and you can=20
>> optimize that really well for all kinds of scenarios.
>>
>> Greets,
>> Jeroen
>
_______________________________________________
Syslog mailing list
Syslog@ietf.org
https://www.ietf.org/mailman/listinfo/syslog