Re: [sip-clf] WGLC: SIPCLFProblemStatement(draft-gurbani-sipclf-problem-statement-01)

"Rainer Gerhards" <> Wed, 03 February 2010 17:08 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id D2C9D28C146 for <>; Wed, 3 Feb 2010 09:08:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.299
X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id h-Vr9e4VAxBk for <>; Wed, 3 Feb 2010 09:08:46 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id 5045E28C172 for <>; Wed, 3 Feb 2010 09:08:46 -0800 (PST)
Received: from localhost (localhost []) by (Postfix) with ESMTP id 2F010241C008; Wed, 3 Feb 2010 17:42:15 +0100 (CET)
Received: from ([]) by localhost (localhost []) (amavisd-new, port 10024) with ESMTP id 8VQ3wSMY6vzo; Wed, 3 Feb 2010 17:42:15 +0100 (CET)
Received: from ( []) by (Postfix) with ESMTP id 9E86D241C005; Wed, 3 Feb 2010 17:42:14 +0100 (CET)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-MimeOLE: Produced By Microsoft Exchange V6.5
Date: Wed, 3 Feb 2010 18:09:26 +0100
Message-ID: <>
Thread-Topic: [sip-clf] WGLC: SIPCLFProblemStatement(draft-gurbani-sipclf-problem-statement-01)
Thread-Index: Acqk8IGV5uGYgRlVRLejHmrwOFjo1QAAEzBg
References: <><00ce01caa41e$fe5a5ef0$><><><01eb01caa4ec$0fbe0930$> <>
From: "Rainer Gerhards" <>
To: "Spencer Dawkins" <>, "David Harrington" <>, "Vijay K. Gurbani" <>, <>
Subject: Re: [sip-clf] WGLC: SIPCLFProblemStatement(draft-gurbani-sipclf-problem-statement-01)
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: SIP Common Log File format discussion list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 03 Feb 2010 17:08:47 -0000

> -----Original Message-----
> From: [] On
> Behalf Of Spencer Dawkins
> Sent: Wednesday, February 03, 2010 5:46 PM
> To: David Harrington; 'Vijay K. Gurbani';
> Subject: Re: [sip-clf] WGLC: SIPCLFProblemStatement(draft-gurbani-
> sipclf-problem-statement-01)
> So, my understanding of the pushback on SYSLOG in Hiroshima - and I
> think
> this is what Vijay was saying in his followup - was that SYSLOG entries
> become really important when Bad Things Happen(TM), and people weren't
> sure
> that producing a lot of SIPCLF SYSLOG entries, which might even be
> message-by-message for a large SIP proxy, that you needed to prune
> through
> in order to find more "interesting" SYSLOG entries like "file system
> full",
> was the right thing to do.
> Since you are a smart guy about SYSLOG - is this a realistic concern
> for us
> to have? and, if the answer is "yes", does modern SYSLOG with
> structured
> data elements provide a way to either (1) quickly separate the SIPCLF
> entries from other "oh, no" SYSLOG entries, or (2) send SIPCLF log
> entries
> using the same protocol to two different destinations, one potentially
> containing massive amounts of SIPCLF log entries, and the other
> containing
> "everything else"?

This is actually done with ancient syslog in practice. Syslog provides a
so-called facility, which can be used as a very rough indication of the
message type. If you dig into the details, there is much that one can argue
about (and the syslog WG has done in length), but properly deployed this is a
very fast and efficient mechanism to separate different types of log sources.
Also, you can send log entries to as many destinations as you like. I myself
are doing this for over 15 years now, and others do it much longer.

A big problem was that traditional syslog mainly used UDP transport and thus
was not well-suited (to phrase it lightly) for important data (but see [1]
below). Nonetheless even financial institutions used (and continue to use)
traditional syslog with great success for audit data.

Modern syslog solves or at least considerably improves on traditional syslog,
by proving much better reliability, and enhanced filtering capabilities. It
should be noted that each of the large syslog implementations (syslog-ng,
rsyslog, WinSyslog, ...) provide very complex filtering rules, and do so for
year, as it is typical that a stream of syslog messages must be dispersed
into separate streams, for example, to log everything to an audit log, feed
specific events to a security console and feed some other events to a billing
system (in the service provider case). 

In my *personal* view, there are three main points that lead to the
overwhelmingly large deployments of syslog:

* ease of implementation (aka support for 
  lazy implementations ;))
* capability to create very flexible 
  filtering and message routing scenarios
* wide availability of components, both servers
  and clients (related to point 1 above ;)))

Please note that syslog has been implemented outside of the pure system
management space. It is being used at least for

* billing purposes (ISPs, other service providers)
* capturing audit data
* storing connection records for 
  legal purposes (at least in Germany)
* IHE (Healthcare) [1]

I am aware of at least one other larger body that expressed interest in using
syslog for application-level data that has *absolutely* nothing to do with IT
management (but cannot tell more about it due to NDA - just let me say that I
was really *surprised* that this was a use case for syslog).

So I think it would be useful to list the requirements, and see if syslog can
provide a solution or not.



> I would really like to use one of the existing management frameworks
> (still
> speaking as a participant here), IFF it's the right thing to do.
> </as-participant>
> Thanks,
> Spencer
> _______________________________________________
> sip-clf mailing list