Re: [Syslog] Small draft for Syslog File Storage?

Simon Josefsson <> Wed, 10 November 2010 11:52 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id EE0D528C0D6 for <>; Wed, 10 Nov 2010 03:52:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -102.599
X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id RpwINkJgX2An for <>; Wed, 10 Nov 2010 03:52:39 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id D754A3A6AA2 for <>; Wed, 10 Nov 2010 03:52:38 -0800 (PST)
Received: from ( []) (authenticated bits=0) by (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id oAABqVss012806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 10 Nov 2010 12:52:32 +0100
From: Simon Josefsson <>
To: "Rainer Gerhards" <>
References: <>
OpenPGP: id=B565716F; url=
Date: Wed, 10 Nov 2010 12:52:34 +0100
In-Reply-To: <> (Rainer Gerhards's message of "Wed, 10 Nov 2010 07:24:17 +0100")
Message-ID: <>
User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Virus-Scanned: clamav-milter 0.96.3 at yxa-v
X-Virus-Status: Clean
X-Mailman-Approved-At: Wed, 10 Nov 2010 08:04:12 -0800
Subject: Re: [Syslog] Small draft for Syslog File Storage?
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Security Issues in Network Event Logging <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 10 Nov 2010 11:52:40 -0000

"Rainer Gerhards" <> writes:

> Hi all,
> In what we did, we specified the on-the-wire format. However, we did not
> specify any format to use when persisting syslog data to a file.
> Note that we were very generous when specifying the on-the-wire format, for
> example we permit LF, CR, NUL and many other characters considered dangerous
> in file formats.
> There are many tools available which interpret syslog data stored in text
> files. However, different syslog implementations may use slightly different
> file formats.
> Together with the control character issue, the file format question both has
> interoperability AND security issues. I think these would be very easy to fix
> if we write a small RFC that specifies how text is to be encoded. It would be
> similar, but much smaller to RFC4627 (JSON). Actually, I think we would need
> to carry over primarily its section 2.5.
> I would volunteer to write an initial draft, but would first like to get some
> feedback if this effort has any chance of getting through.

I would support that effort.  Multiple incompatible syslog formats are a
pain, and it has bitten me several times in multiple jobs.

One approach that would be easy to move forward with is that you write a
document registering a MIME media sub-type, e.g., text/syslog or
application/syslog.  Describing one reasonable syslog text format in
that document.  With this approach, there is less pressure on all syslog
vendors to agree on a particular format, and you can just invent one
format that makes sense and let people adopt it on an opt-in basis.
I've registered some MIME media types, and learned some details when
doing that, so I could help with this approach if you want.  Having a
MIME media type specified would also enable reliable transfer of syslog
data in protocols that are MIME aware (e.g., HTTP or e-mail).  With this
approach, there is also less of a requirement to be backwards compatible
with existing (often sub-optimal) formats.