RE: [Syslog] byte-counting vs special character

Carson Gaspar <carson@taltos.org> Tue, 15 August 2006 20:48 UTC

Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1GD5qA-0003OQ-Uz; Tue, 15 Aug 2006 16:48:46 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1GD5qA-0003OD-0a for syslog@ietf.org; Tue, 15 Aug 2006 16:48:46 -0400
Received: from dsl081-242-052.sfo1.dsl.speakeasy.net ([64.81.242.52] helo=gandalf.taltos.org) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1GD5q9-0000iU-AH for syslog@ietf.org; Tue, 15 Aug 2006 16:48:45 -0400
Received: from gandalf.taltos.org (localhost [127.0.0.1]) by gandalf.taltos.org (Postfix) with ESMTP id 894ED21C14 for <syslog@ietf.org>; Tue, 15 Aug 2006 13:48:38 -0700 (PDT)
X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on gandalf.taltos.org
X-Spam-Level:
X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.0
Received: from [192.168.1.2] (unknown [192.168.1.2]) by gandalf.taltos.org (Postfix) with ESMTP id 872A121B73 for <syslog@ietf.org>; Tue, 15 Aug 2006 13:48:38 -0700 (PDT)
Date: Tue, 15 Aug 2006 13:50:40 -0700
From: Carson Gaspar <carson@taltos.org>
To: syslog@ietf.org
Subject: RE: [Syslog] byte-counting vs special character
Message-ID: <C136C3E9AA2A5B0FE58F3084@[192.168.1.2]>
In-Reply-To: <98AE08B66FAD1742BED6CB9522B7312201C971AC@xmb-rtp-20d.amer.cisco.com>
References: <98AE08B66FAD1742BED6CB9522B7312201C971AC@xmb-rtp-20d.amer.cisco .com>
X-Mailer: Mulberry/4.0.4 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format="flowed"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Score: 0.1 (/)
X-Scan-Signature: c2e58d9873012c90703822e287241385
Cc:
X-BeenThere: syslog@lists.ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Security Issues in Network Event Logging <syslog.lists.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@lists.ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/syslog>
List-Post: <mailto:syslog@lists.ietf.org>
List-Help: <mailto:syslog-request@lists.ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/syslog>, <mailto:syslog-request@lists.ietf.org?subject=subscribe>
Errors-To: syslog-bounces@lists.ietf.org

I've been mostly lurking lately, but I would like to re-iterate my support 
for byte counting. It's the Right Thing To Do, and theoretical "backwards 
compatibility" with existing non-standard non-widely-deployed syslog over 
TCP solutions isn't a good enough reason to put bad ideas in the standard.

--On Tuesday, August 15, 2006 2:04 PM -0400 "Anton Okmianski (aokmians)" 
<aokmians@cisco.com> wrote:

> I second these concerns.  Escaping requirements result in a more
> interdependent layering, which is a weaker architecture (not to mention
> the overhead to a new standard). The syslog transport would need to mess
> with payload instead of treating it as opaque blob with easily known
> length. Not nice. Imagine TCP/IP escaping all payload to separate
> datagrams and segments.
>
> Escaping of magic characters is IMHO clearly a hack and should not be
> put into a standard! If the argument was to accommodate some real
> established standard and there was no way to version things - maybe.
> Current syslog is not standardized (not in IETF, not in reality). The
> transition to a cleaner design is trivial.  A given implementation can
> support whatever legacy compatibility modes it wishes to support. Syslog
> is sent to a known destination configured by administrator, not some
> kind of broadcast to world-wide-web where you don't know what kind of
> receiver will get it. This problem is easily administrated in mixed
> legacy/standard environment if someone chooses to deploy one.
>
> Anton.
>
>> -----Original Message-----
>> From: David Harrington [mailto:ietfdbh@comcast.net]
>> Sent: Tuesday, August 15, 2006 1:25 PM
>> To: 'Rainer Gerhards'
>> Cc: syslog@ietf.org
>> Subject: [Syslog] byte-counting vs special character
>>
>> Hi Rainer,
>>
>> [speaking as co-chair]
>> Can we change the subject line to "byte-counting vs special character"
>> for this topic so it is easier to find comments on this topic
>> as compared to other things in the timeline? That will make
>> it nuch easier for the chairs.
>>
>> This WG has already gotten stuck, and had WG progress stall,
>> trying to provide backwards compatibility to a bunch of
>> incompatible implementations. I argued on this list before
>> becoming co-chair that backwarsd compatibility may not be
>> achieveable for some features and we need to stop getting
>> hung up on it. Sometimes to build a good standard, you need
>> to choose something that will break some existing implementations.
>>
>> I raised this concern with Chris when I started as co-chair.
>> I do not want to see backwards compatibility arguments stall
>> progress again. I made sure this was reflected in the
>> timeline, which says that by Friday (if you decide to use a
>> special character) you must reach consensus on which special
>> character to use.
>> [/speaking as co-chair]
>>
>> [speaking as contributor]
>> I like the argument that the LF solution will not break
>> existing implementations, but I would like to know this is
>> actually true. Have you actually tested this against multiple
>> implementations, or is it a theoretical argument?
>>
>> I know you have tested a number of other proposed ways of
>> doing things against multiple implementations to try to
>> verify backwards compatibility. Have you actually tested
>> multiple existing implementations with the LF and found that
>> they do continue to work without significant problems? Can
>> you tell the WG which ones you have tested? Are there
>> implementations that break when using this solution?
>>
>>
>> In a different message you argue that syslog-sign only needs
>> to support -protocol because the implementations will need to
>> have new code added anyway, and the added complexity of
>> backwards compatibility to rfc3164 implementations is not needed.
>>
>> You only mention one implementation explicitly that provides
>> syslog/tls, syslog-ng over stunnel. Would all the other
>> implementations need to be modified to support a TLS
>> substrate anyway, so it would not make a difference to them
>> which special character was chosen or if byte-counting was
>> chosen? Which syslog/tls implementations will be impacted by
>> our choice here?
>>
>> Ideally, only the implementations that support a feature need
>> to pay the price for the feature.
>>
>> A syslog message delineator will only be needed in
>> implemntations that add support for syslog/tls (or other
>> streams). Whether one supports byte-counting or a special
>> character, only implementations that support multiple
>> messages in a stream, e.g. syslog/tls, should need to be
>> modified to support the choice.
>>
>> Personally I find the octet-counting cleaner because previous
>> discussions on terminators showed that available
>> implementations are highly inconsistent with their special
>> characters.
>>
>> Since byte-counting happens outside the syslog message, and
>> only for syslog in a delimited stream coming in over the
>> syslog/tls port, it would not impact existing code, only code
>> that supports syslog/tls.
>> New syslog/tls code would be needed to extract each "normal"
>> syslog message from the stream and then send it on for
>> processing through the existing parser.
>>
>> If message originators start escaping special characters in
>> the message content, existing receivers may need to be
>> modified to un-escape the characters. Think about
>> applications, such as intrusion detection systems, that
>> search for special patterns in syslog content
>> - those could be broken by having the content changed by
>> escaping characters in the middle of the content.
>>
>> What happens at a store-then-forward relay? Does the relay
>> store the escaped characters or does it un-escape them first?
>> Does it escape them again when it forwards the message? Does
>> it escape them if it un-escaped them on input? Does it escape
>> them if it did not un-escape them on input? What if the
>> sender included some escaped characaters (like '\') for other
>> purposes? Should the relay unescape them as well?
>> Will it know to re-escape them on output? What should the
>> ultimate reciever expect to receive?
>>
>> Byte-counting looks way less complex to me than choosing a
>> special character and escaping/unescaping special characters
>> in the content during the sending/forwarding/recieving
>> process. (And my opinion has nothing to do with the H**** in
>> my email address.)
>>
>> dbh
>>
>> > -----Original Message-----
>> > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com]
>> > Sent: Tuesday, August 15, 2006 12:41 AM
>> > To: Miao Fuyou
>> > Cc: syslog@ietf.org
>> > Subject: RE: [Syslog] timeline
>> >
>> > Miao,
>> >
>> > I am actually concerned about backward compatibility with existing
>> > code
>> > *without* the need to upgrade any of that code. As you know,
>> deployed
>> > software tends to stick.
>> >
>> > If we use just LF, existing, deployed technology (e.g. syslog-ng
>> with
>> > stunnel) would be able to understand a message sent from a "new
>> style"
>> > syslogd. Having the octet count in front of the message
>> removes that
>> > ability, as the old syslogd will no longer see the <pri> at
>> the start
>> > of the message.
>> >
>> > I agree that it is trivial to modify code to take care for
>> the octet
>> > counter. But this is not my concern. My concern is that I
>> would like
>> > to achive as good as possible compatibility with existing deployed
>> > (aka
>> > "unmodified") technology. I should have been more specific on that.
>> > Sorry for the omission...
>> >
>> > I am also unaware of any implementation that mandates CR LF over
>> just
>> > LF. Could you let me know which ones are these?
>> >
>> > Rainer
>> >
>> > > -----Original Message-----
>> > > From: Miao Fuyou [mailto:miaofy@huawei.com]
>> > > Sent: Monday, August 14, 2006 7:07 PM
>> > > To: Rainer Gerhards
>> > > Cc: syslog@ietf.org
>> > > Subject: RE: [Syslog] timeline
>> > >
>> > >
>> > > Hi, Rainer,
>> > >
>> > > A new implementation could rely on byte-counting only and then
>> > > delete LF from the frame(appplication knows exactly where the LF
>> > > is), it may not force us to use escapes. For LF, I think it is
>> > > difficult to get 100% compatibility for a legacy
>> implementation to
>> > > comply TLS-transport without any change to the code. At
>> least, some
>> > > imlementation may need to change
>> > CR LF to LF
>> > > because some implementations use CR LF rather than LF.
>> So, it may be
>> > > ok to add several LOC to delete FRAME-LEN SP from the frame.
>> > >
>> > > I still prefer byte-counting only to byte-counting+LF even
>> > if it is a
>> > > feasible tradeoff.
>> > >
>> > > Miao
>> > >
>> > > > -----Original Message-----
>> > > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com]
>> > > > Sent: Monday, August 14, 2006 10:18 PM
>> > > > To: Miao Fuyou
>> > > > Subject: RE: [Syslog] timeline
>> > > >
>> > > > We should not go byte-counting + LF. This is the worst choice:
>> it
>> > > >
>> > > > A) breaks compatibility
>> > > > B) Forces us to use escapes
>> > > >
>> > > > So we get the bad of both worlds, without any benefits.
>> > > >
>> > > > Rainer
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: Miao Fuyou [mailto:miaofy@huawei.com]
>> > > > > Sent: Monday, August 14, 2006 12:58 AM
>> > > > > To: 'Anton Okmianski (aokmians)'; 'David Harrington';
>> > > > syslog@ietf.org
>> > > > > Subject: RE: [Syslog] timeline
>> > > > >
>> > > > >
>> > > > > My vote: byte-counting only > byte-counting + LF > LF
>> > > >
>> > > >
>> > >
>> > >
>> > >
>> >
>> > _______________________________________________
>> > Syslog mailing list
>> > Syslog@lists.ietf.org
>> > https://www1.ietf.org/mailman/listinfo/syslog
>> >
>>
>>
>> _______________________________________________
>> Syslog mailing list
>> Syslog@lists.ietf.org
>> https://www1.ietf.org/mailman/listinfo/syslog
>>
>
> _______________________________________________
> Syslog mailing list
> Syslog@lists.ietf.org
> https://www1.ietf.org/mailman/listinfo/syslog

_______________________________________________
Syslog mailing list
Syslog@lists.ietf.org
https://www1.ietf.org/mailman/listinfo/syslog