Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?

Jeroen Massar <jeroen@unfix.org> Wed, 16 February 2011 12:53 UTC

Return-Path: <jeroen@unfix.org>
X-Original-To: syslog@core3.amsl.com
Delivered-To: syslog@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5D29E3A6CAA; Wed, 16 Feb 2011 04:53:58 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.297
X-Spam-Level:
X-Spam-Status: No, score=-102.297 tagged_above=-999 required=5 tests=[AWL=0.302, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xjbNwU+UPS5a; Wed, 16 Feb 2011 04:53:55 -0800 (PST)
Received: from abaddon.unfix.org (abaddon.unfix.org [62.220.146.203]) by core3.amsl.com (Postfix) with ESMTP id 896D13A6CC6; Wed, 16 Feb 2011 04:53:54 -0800 (PST)
Received: from [IPv6:2001:41e0:ff42:99:222:cfff:fe31:ce41] (spaghetti.ch.unfix.org [IPv6:2001:41e0:ff42:99:222:cfff:fe31:ce41]) (using SSLv3 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: jeroen) by abaddon.unfix.org (Postfix) with ESMTPSA id BEAC621778; Wed, 16 Feb 2011 13:53:52 +0100 (CET)
Message-ID: <4D5BC8F8.70008@unfix.org>
Date: Wed, 16 Feb 2011 13:54:16 +0100
From: Jeroen Massar <jeroen@unfix.org>
Organization: Unfix
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
References: <4D5A60C8.3090000@unfix.org><93ED0A84F9A1D74FA65021D940AA588405446C41F9@IMCMBX3.MITRE.ORG> <4D5BA85B.7040007@unfix.org> <9B6E2A8877C38245BFB15CC491A11DA71DDC71@GRFEXC.intern.adiscon.com> <4D5BAD69.2060608@unfix.org> <9B6E2A8877C38245BFB15CC491A11DA71DDC72@GRFEXC.intern.adiscon.com>
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDC72@GRFEXC.intern.adiscon.com>
X-Enigmail-Version: 1.1.1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc: cee@mitre.org, syslog@ietf.org, sip-clf@ietf.org
Subject: Re: [Syslog] draft-cloud-log-00 / CEE - why not IPFIX?
X-BeenThere: syslog@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Security Issues in Network Event Logging <syslog.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/syslog>
List-Post: <mailto:syslog@ietf.org>
List-Help: <mailto:syslog-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 16 Feb 2011 12:53:58 -0000

On 2011-02-16 12:34, Rainer Gerhards wrote:
[..]
> The question is if such an effort must be bound restricted to a single
> protocol. My PoV is that this is counter-productive.

I might be missing the parsing here, but can you explain what part is
counter-productive to have a single protocol and having to write only a
single parser versus having a lot of different protocols who require
parsers which can resolve ambiguity?

> You definitely have a point in that IPFIX may be superior than syslog in many
> regards. I do not intend to argue against this. But often a simpler solution
> is able to draw more attention, and thus deployments, than a (potentially or
> actually) technical superior one (shouldn't we all use the OSI stack by now,
> just as one example...). 

I wonder what the "E" in IETF stands for if I see the above statement.
Why not simply use CSV or hey just pack it in XML! :)

Please note that IPFIX has gathered quite some attention already.

As for the OSI stack argument, the world is not ready for IPv6 either.

I like to also quote:
http://www.ietf.org/mail-archive/web/sip-clf/current/msg00430.html

8<-----------------------------------------------------------------------------
1. Shall we use a TAB or a SPACE (or any LWS) as field delimiters?
Considerations and points raised:

- TABs don’t survive Telnet or web pages very well, especially when
copy/pasting.
- spaces can appear inside fields (as can most other delimiters we choose)
- how do TAB and SPACE delimiters interact with shell tools (rep, cut, awk)
----------------------------------------------------------------------------->8

The moment you have to worry about a delimiter, you have big
problems.... that is why char format is not a good idea. Yes, you can
use all the standard unix command tools to process them, but are you
really going to do this on the big scale? I certainly hope not.

The same with Apache logs, very useful to have to parse the IP address
again to get the information you want. Especially with IPv6 where the
address can have quite a number of formats based on how the address is
represented, not even going talking about the point.

Oh and those unix tools, they do not support UTF-8 in a lot of cases,
thus they are suddenly quite useless.

> I don't think it is useful to include IPFIX in syslog. But it may be an
> option that IPFIX makes syslog obsolete. I think you should take that later
> route.

If there is going to be new effort for syslog, is that then not the
route it should be taking? Structured data is so much more useful than
unstructured data, and that is the big problem that

> But as I said -- I do not intend to spawn another iteration of this lengthy
> discussion. It has occurred sooo often in the past years.

Is there a summary of this discussion with a clear listing of pro/cons?
Would be good to have that in a draft, is there one? If there isn't
please write one with those arguments so that it can referred to instead
of stating 'we did this before, EOD'...

Greets,
 Jeroen