Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
"Heinbockel, Bill" <heinbockel@mitre.org> Thu, 17 February 2011 16:36 UTC
Return-Path: <heinbockel@mitre.org>
X-Original-To: syslog@core3.amsl.com
Delivered-To: syslog@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 17B5D3A6C7D for <syslog@core3.amsl.com>; Thu, 17 Feb 2011 08:36:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ClNBL3UcbgSy for <syslog@core3.amsl.com>; Thu, 17 Feb 2011 08:36:01 -0800 (PST)
Received: from smtpksrv1.mitre.org (smtpksrv1.mitre.org [198.49.146.77]) by core3.amsl.com (Postfix) with ESMTP id AC3AD3A6C3C for <syslog@ietf.org>; Thu, 17 Feb 2011 08:36:01 -0800 (PST)
Received: from smtpksrv1.mitre.org (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id A1DBC21B17F3; Thu, 17 Feb 2011 11:36:32 -0500 (EST)
Received: from imchub2.MITRE.ORG (imchub2.mitre.org [129.83.29.74]) by smtpksrv1.mitre.org (Postfix) with ESMTP id 8F71321B17D3; Thu, 17 Feb 2011 11:36:32 -0500 (EST)
Received: from IMCMBX3.MITRE.ORG ([129.83.29.206]) by imchub2.MITRE.ORG ([129.83.29.74]) with mapi; Thu, 17 Feb 2011 11:36:32 -0500
From: "Heinbockel, Bill" <heinbockel@mitre.org>
To: Jeroen Massar <jeroen@unfix.org>
Date: Thu, 17 Feb 2011 11:35:19 -0500
Thread-Topic: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
Thread-Index: AcvNxSVcMeG8rWBOQCCWiPHcleUWdwA+p2yQ
Message-ID: <93ED0A84F9A1D74FA65021D940AA5884054477F158@IMCMBX3.MITRE.ORG>
References: <4D5A60C8.3090000@unfix.org> <93ED0A84F9A1D74FA65021D940AA588405446C41F9@IMCMBX3.MITRE.ORG> <4D5BA85B.7040007@unfix.org>
In-Reply-To: <4D5BA85B.7040007@unfix.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/signed; micalg="SHA1"; protocol="application/x-pkcs7-signature"; boundary="----=_NextPart_000_0033_01CBCE96.C12A9240"
MIME-Version: 1.0
Cc: Sam Johnston <sj@google.com>, Event Expression <cee@mitre.org>, "syslog@ietf.org" <syslog@ietf.org>, Common
Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
X-BeenThere: syslog@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Security Issues in Network Event Logging <syslog.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/syslog>
List-Post: <mailto:syslog@ietf.org>
List-Help: <mailto:syslog-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Feb 2011 16:36:04 -0000
Thanks for that, I did not realize that IPFIX had been extended beyond its netflow past. I don't have the time now, but I am interested in looking into it further. It does kind of remind me of ASN.1/SNMP, where we need to worry about the names/OID translation Also, a lot of vendors and users seem to prefer the ease of text-based protocols like Syslog for logging. I am not saying this is good or bad, but it seems to be the sweetspot -- binary is not natively human readable and XML has too much overhead. William Heinbockel The MITRE Corporation >-----Original Message----- >From: Jeroen Massar [mailto:jeroen@unfix.org] >Sent: Wednesday, 16 February 2011 05:35 >To: Heinbockel, Bill >Cc: syslog@ietf.org; Chris Lonvick; Gene Golovinsky; Sam Johnston; Common >Event Expression >Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX? > >On 2011-02-16 06:21, Heinbockel, Bill wrote: >> From what I understand, IPFIX is for expression of IP flows from network >sensing >> devices. > >For a short bit forget about the history of IPFIX, it indeed comes from >NetFlow, and thus is used quite in a network centric way, but >effectively it is a structured streaming data format. > >> Could you please explain how IPFIX is relevant to event and cloud logging >data? >> I understand how CEE and IPFIX may overlap for describing networking >events, but >> it is unclear to me how IPFIX could handle things like Windows Event Logs >and >> RHEL audit logs. > >There are two parts to IPFIX: Templates + Data > >The template describes how the data looks like, for instance, lets take >an Apache CLF log entry: > >66.249.66.174 - - [16/Feb/2011:10:48:11 +0100] "GET /robots.txt >HTTP/1.1" 200 2629 "-" "Googlebot-Image/0" > >We can make an IPFIX template for that > >[ > {4, IPv4_SRC }, > {4, TIMESTAMP}, > {4, HTTP_METHOD}, > {v, URL}, > {v, HTTP_PROTOCOL}, > {2, HTTP_RESULT}, > {8, OCTETS}, > {v, HTTP_REFER}, > {v, HTTP_USERAGENT}, >] > >The 'v' markers indicate variable fieldlengths, the others indicates the >number of bytes such a field takes. The data is then just encoded in the >above format, presto. > >The above is a simple example, one can also have repeating lists and of >course you could make a variable template which just includes the fields >that you actually want to look at or you could already do some >aggregation and add other fields. Templates are only sent every now and >then, as they should not change. The data is the important bit. > >The fieldnames are actually numbers in the data, thus very compact, and >are mapped to descriptions, data types etc, per a nice XML file > http://www.iana.org/assignments/ipfix/ipfix.xml (or .xhtml or .txt for >a more human readable version ;) for the official IANA list and with the >help of Enterprise IDs any others can easily be added. > >The big advantage is that you can more or less do static templates if >you want and you only need one single parser on the collector side, thus >one does not have to create another parser and collector again for >decoding other protocols, just one, the IPFIX one, and you can optimize >that really well for all kinds of scenarios. > >Greets, > Jeroen
- [Syslog] draft-cloud-log-00 / CEE - why not IPFIX? Jeroen Massar
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Jeroen Massar
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Rainer Gerhards
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Jeroen Massar
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Rainer Gerhards
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Jeroen Massar
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Heinbockel, Bill
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Dan Schlitt
- Re: [Syslog] draft-cloud-log-00 / CEE - why not I… Gene Golovinsky