Re: [Json] JSON Log file encoding JSON-L

Tim Bray <tbray@textuality.com> Tue, 06 May 2014 14:20 UTC

Return-Path: <tbray@textuality.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 034621A032F for <json@ietfa.amsl.com>; Tue, 6 May 2014 07:20:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.977
X-Spam-Level:
X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ymcaRvko18Qu for <json@ietfa.amsl.com>; Tue, 6 May 2014 07:20:36 -0700 (PDT)
Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by ietfa.amsl.com (Postfix) with ESMTP id DAD731A029E for <json@ietf.org>; Tue, 6 May 2014 07:20:35 -0700 (PDT)
Received: by mail-vc0-f178.google.com with SMTP id hu19so1021409vcb.37 for <json@ietf.org>; Tue, 06 May 2014 07:20:31 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=o8YzulRxvkXiUknww7W/VXHuilX1T4vb+ZEex48d6io=; b=DoTHw36p5ko4kpzu4w2Pad+PObgJLKTymVwC+Mb3RRwOVmqezahQzj2fGWOUbHc91e HCwTrFwUrmlPjrQUBNz10WLOD8wVyCF4l7vkEnVReQyaqM39c5qLQyYh0BJ4s+NKiPDz S3Xo3u88q1PtqJQRANa5nFXFURyPFi0fYmbBp8FoX69qq7R4+Xc9OcQvg7GJiTJCcCIL gJh6wBsC91SQHcW0fg7OU5tOj6jNK4PY7JFt+V96MhRxfMMHHQdc71V5XmZRwb/AquGh XRq4jCu4xfgOkSts8TtpUxOrSkCAs5c+kQdIyMbKfUYtoHf0K+n3j2eMFJM4SEbQKFYS xiSA==
X-Gm-Message-State: ALoCoQkij99FaFbb/UEVPs/0TxkFUaC/doGKP/7i7/g3tImHtwoOPlg5E2QIJEtATvZy8mIjnKTP
MIME-Version: 1.0
X-Received: by 10.58.126.4 with SMTP id mu4mr33998994veb.0.1399386031766; Tue, 06 May 2014 07:20:31 -0700 (PDT)
Received: by 10.220.98.73 with HTTP; Tue, 6 May 2014 07:20:31 -0700 (PDT)
X-Originating-IP: [24.84.235.32]
Received: by 10.220.98.73 with HTTP; Tue, 6 May 2014 07:20:31 -0700 (PDT)
In-Reply-To: <CAK3OfOgX06vkOS1+CJhsptMAvm+KX1HgB2=34ubxUCtHNVnx=A@mail.gmail.com>
References: <CAMm+LwjB-51z4GoeC0riehmJg1HAddmLAyfMsVOCVM80i=RiMA@mail.gmail.com> <CAK3OfOgX06vkOS1+CJhsptMAvm+KX1HgB2=34ubxUCtHNVnx=A@mail.gmail.com>
Date: Tue, 06 May 2014 07:20:31 -0700
Message-ID: <CAHBU6is7g6ecwupv-N7wW+VTN_bZL71zbHYj=ePq=iqmk=gUwQ@mail.gmail.com>
From: Tim Bray <tbray@textuality.com>
To: Nico Williams <nico@cryptonector.com>
Content-Type: multipart/alternative; boundary="047d7b6700af53e68704f8bbf34e"
Archived-At: http://mailarchive.ietf.org/arch/msg/json/xM1J4mHor8-td3N05faWJlqFTmg
Cc: Phillip Hallam-Baker <hallam@gmail.com>, json@ietf.org
Subject: Re: [Json] JSON Log file encoding JSON-L
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 May 2014 14:20:39 -0000

If you want to use line-oriented tools, don't use JSON.  The no-newlines
rule seems really outside the spirit of JSON.
On May 6, 2014 12:01 AM, "Nico Williams" <nico@cryptonector.com> wrote:

> For a variety of reasons I would prefer that a log file format based
> on JSON simply forbid the appearance of unescaped newlines in JSON
> texts, and otherwise be just JSON sequences:
>
>  - it's compatible with JSON sequences
>  - it works with line-oriented Unix tools (e.g., wc(1), grep(1), ...)
>  - aesthetically it's nice
>  - it's simpler to resync: just search for the newlines (or EOF, or
> offset 0) around entries that fail to parse
>
>  - it's easy to implement because most JSON encoders I've seen (all?
> I think so) have an option for producing "compact" output,
> usually/always meaning no whitespace will be emitted between the
> tokens that make up a JSON text -- just use this feature and that's
> that
>
>  - it even works without compact JSON encoding if you're lucky to not
> have crashes and such
>
>    Some OSes don't guarantee that writes to files will be completed
> when a process is SIGKILLed.  E.g., recent Linux kernels don't
> guarantee that more than one byte will be written in that case -- this
> was well-intentioned in that a process could start an enormous write
> that an admin might want to stop, but how would they? which is why
> Linux now allows SIGKILL to terminate incomplete writes.  Therefore
> power failures and system crashes are not the only thing to worry
> about.
>
>    Even with non-compact entry encodings, one can always recover with
> some heuristics, and at some cost.  The biggest problem here is the
> SIGKILL problem.
>
> Nico
> --
>
> _______________________________________________
> json mailing list
> json@ietf.org
> https://www.ietf.org/mailman/listinfo/json
>