[sip-clf] A primer on syslog.

"David Harrington" <ietfdbh@comcast.net> Wed, 03 February 2010 19:49 UTC

Return-Path: <ietfdbh@comcast.net>
X-Original-To: sip-clf@core3.amsl.com
Delivered-To: sip-clf@core3.amsl.com
Received: from localhost (localhost []) by core3.amsl.com (Postfix) with ESMTP id EA61B28C104 for <sip-clf@core3.amsl.com>; Wed, 3 Feb 2010 11:49:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599]
Received: from mail.ietf.org ([]) by localhost (core3.amsl.com []) (amavisd-new, port 10024) with ESMTP id 4LzhMve8WVOZ for <sip-clf@core3.amsl.com>; Wed, 3 Feb 2010 11:49:25 -0800 (PST)
Received: from qmta05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net []) by core3.amsl.com (Postfix) with ESMTP id 1C6703A67B4 for <sip-clf@ietf.org>; Wed, 3 Feb 2010 11:49:25 -0800 (PST)
Received: from omta12.westchester.pa.mail.comcast.net ([]) by qmta05.westchester.pa.mail.comcast.net with comcast id dQ2S1d0070xGWP855XpPad; Wed, 03 Feb 2010 19:49:23 +0000
Received: from Harrington73653 ([]) by omta12.westchester.pa.mail.comcast.net with comcast id dXq81d00h284sdk3YXq8ui; Wed, 03 Feb 2010 19:50:09 +0000
From: "David Harrington" <ietfdbh@comcast.net>
To: "'Spencer Dawkins'" <spencer@wonderhamster.org>, "'Vijay K. Gurbani'" <vkg@alcatel-lucent.com>, <sip-clf@ietf.org>
References: <7505A2C58D8F4FD88B47D10EA74649CD@china.huawei.com><00ce01caa41e$fe5a5ef0$0600a8c0@china.huawei.com> <4B68920B.5090908@alcatel-lucent.com> <81F3AF977ADD49E7A883F138ECCC71D0@china.huawei.com> <01eb01caa4ec$0fbe0930$0600a8c0@china.huawei.com> <DCBC0046E3434E519A2E9EDB4071C7B6@china.huawei.com>
Date: Wed, 3 Feb 2010 14:50:07 -0500
Message-ID: <023501caa50a$16943c70$0600a8c0@china.huawei.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Office Outlook 11
In-Reply-To: <DCBC0046E3434E519A2E9EDB4071C7B6@china.huawei.com>
Thread-Index: Acqk8HGw9hmy+vMRTBW+9/JjIJoFBgADdXdw
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198
Subject: [sip-clf] A primer on syslog.
X-BeenThere: sip-clf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: SIP Common Log File format discussion list <sip-clf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sip-clf>
List-Post: <mailto:sip-clf@ietf.org>
List-Help: <mailto:sip-clf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sip-clf>, <mailto:sip-clf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Feb 2010 19:49:27 -0000


For those unacquainted with syslog, let me try to summarize how it
works. I will focus on the forwarding and filtering and security
mechanisms, since I think those will be important considerations for
SIP logging.

Rainer is the guy I go to when I have a question about syslog. He has
answered Vijay's concern in considerable detail in another email. I
don't personally deploy syslog, so Rainer can feel free to correct
anything I get wrong. ;-)

I will recommend looking at the wikipedia entry, which has a summary
of its history and usage, but some factual details about the protocol
are about the limitations of BSD syslog, rather than the IETF standard
syslog. As technical advisor, I recommend the IETF standard syslog
because it overcomes many of the limitations and interoperability
issues with BSD syslog. The wikipedia entry mentions some work that is
being done, work that relates to anomaly detection and security
management systems.

The IETF has worked at standardizing syslog across operating systems,
and focused mainly on standardizing aspects of syslog that impacted
security and filtering and correlation (such as standardizing the
header, and more detailed info about the source and time of the

---- The forwarding mechanism ----

Syslog allows log information for both kernel processes and user
processes to be forwarded to a file, a set of users, or another node
in a network. 

IETF syslog provides support for UDP, TLS, and (soon) DTLS. It is
designed to allow additional transports to be defined, so could also
be used in non-IP environments, I suppose. 

This flexibility in the forwarding rules allows syslog to concentrate
on collecting data and putting it where the operator wants it so other
applications can focus on sorting, filtering, and alerting on the
contents. Forwarding to a collector elsewhere in the network can help
prevent attackers from modifying the local logs to cover their tracks,
and allows a centralized location for operators and tools to do log
analysis rather than having to go to each device to look at the logs.
But you can also store it locally if you want.

---- The filtering mechanism ----

syslog messages start with a <priority> byte that is calculated from
two factors - the facility that generated the message and the severity
of the message. This is the primary mechanism for sorting messages
that are logged.

Facilities define the source of a syslog message. There are 23
different facilities defined for use in the <priority field>, ranging
from "kernel" to system daemons to mail to security to local
applications. A SIP application would use one of the "local"
facilities to log messages. If there are multiple SIP applications,
they might use different "local" facilities to generate separate
logging streams for forwarding purposes, but whether this is done
depends on the implementations and operator configurations. Additional
header fields and the SDE feature of IETF syslog allow much finer
filtering during log analysis.

There are 7 levels of severity ranging from emergency to critical to
informational to debug messages. The IETF standard syslog severities
can be aligned with the ITU-T values of perceived severity (M.2100,
X.733, X.736), and with the SNMP values defined in the ALARM-MIB. This
helps operators correlate events reported by different applications or
by different mechanisms.

syslog.conf is the normal mechanism for configuring what to log and
how to distribute it. The filtering/forwarding instructions look
something like this:

mail.info	/var/log/maillog	-- send all informational
mail-related messages
                                 to the file /var/log/maillog
cron.*      /var/log/cron     -- send all messages from cron,
regardless of
                                 severity to the file /var/log/cron
security.*  @loghost.netlab.lander.edu
                              -- send all security related messages,
                                 of severity to a remote system
*.notice     root             -- send all notices, regardless of
source, to 
                                 the root user, if root is logged on.

and so on.

These are all configured administratively. Operators are usually very
acquainted with syslog as a tool. 

---- Post logging analysis ----

Other header values can also be used for analyzing logs, including
timestamp, hostname, application name, process ID, etc. Filtering
based on header values has been in wide use for a long time, although
the header format for different syslog implementations vary widely.
IETF syslog standardizes the order and format of header fields, but
still keeps roughly the same historic header fields. There are many
tools available for syslog log analysis.

The Structured Data Elements provide even more capability. This is
only available in the IETF standard syslog. My earlier message pointed
out some of the already-standardized SDEs defined by the WG that can
be used to filter log entries.

Hope this helps,

> -----Original Message-----
> From: Spencer Dawkins [mailto:spencer@wonderhamster.org] 
> Sent: Wednesday, February 03, 2010 11:46 AM
> To: David Harrington; 'Vijay K. Gurbani'; sip-clf@ietf.org
> Subject: Re: [sip-clf] WGLC: SIPCLF 
> ProblemStatement(draft-gurbani-sipclf-problem-statement-01)
> Hi, Dave,
> <as-co-chair>
> Thanks for clarifying your role - it's helpful.
> </as-co-chair>
> <as-participant>
> On one specific point:
> >
> > As I read the problem-statement-01, my
> > impression is syslog and/or ipfix might fit well to meet certain
> > goals, however, it is highly possible that syslog and ipfix 
> are WRONG
> > to solve the sipclf problem.
> >
> > As Technical Advisor - not as technical contributor - I 
> recommend that
> > the WG consider syslog and ipfix seriously. My advice as Technical
> > Advisor is to do a real analysis about the applicability of
> > standards like syslog and ipfix to the various sip clf use 
> cases. This
> > should not be just a cursory analysis; we spent probably 
> less than one
> > minute talking about syslog at the last IETF meeting.
> I think the discussion lasted longer than a minute, but I 
> suspect the longer 
> time I'm remembering included people walking to the mike :D
> So, my understanding of the pushback on SYSLOG in Hiroshima - 
> and I think 
> this is what Vijay was saying in his followup - was that 
> SYSLOG entries 
> become really important when Bad Things Happen(TM), and 
> people weren't sure 
> that producing a lot of SIPCLF SYSLOG entries, which might even be 
> message-by-message for a large SIP proxy, that you needed to 
> prune through 
> in order to find more "interesting" SYSLOG entries like "file 
> system full", 
> was the right thing to do.
> Since you are a smart guy about SYSLOG - is this a realistic 
> concern for us 
> to have? and, if the answer is "yes", does modern SYSLOG with 
> structured 
> data elements provide a way to either (1) quickly separate 
> entries from other "oh, no" SYSLOG entries, or (2) send 
> SIPCLF log entries 
> using the same protocol to two different destinations, one 
> potentially 
> containing massive amounts of SIPCLF log entries, and the 
> other containing 
> "everything else"?
> I would really like to use one of the existing management 
> frameworks (still 
> speaking as a participant here), IFF it's the right thing to do.
> </as-participant>
> Thanks,
> Spencer