Re: [Json] JSON Sequence support for log files

["Manger, James" <[e-mail]>]

>>> Appending to logfiles creates the need to recover from interrupted
>>> writes (i.e., corrupted entries).  Therefore I propose the following
>>> minor change to the JSON Sequence I-D:
>>>
>>> a) add ASCII RS to the whitespace rule
>>> b) RECOMMEND that logfile writers write either RS <entry> RS NL or
>>> <entry-without-internal-newlines> NL
>>>
>>> (producing entries with no internal newlines is trivial, as discussed
>>> in other posts)
>>>
>>> c) describe how to resynchronize by seeking to the next RS or NL when
>>> a JSON text fails to parse, until a text that parses is found.

>> Why can't we have } NL { as a record boundary separator?

> JSON Sequence is more general than logfiles.  As such it supports any
> type at the top-level, therefore } NL { doesn't work.
>
> For logfiles, however, it does work, as we can constrain the type used
> at the top-level in logfiles.
>
> In fact, any sequence of ( %x22 / "]" / "}" ) %x0A ( %22 / "{" / "[" )
> works!  Thus even for logfiles we can have arrays, objects, and
> strings at the top-level.


Syncing to a boundary in the middle of a JSON sequence or log isn't a good reason for RS. You can sync without RS. I think the following detects a boundary between any JSON values.

( "}" / "]" / %x22 / "e" / "l" / DIGIT ) *ws NL *ws ( "{" / "[" / %x22 / "t" / "f" / "n" / "-" / DIGIT )

RS adds complexity (eg extra steps to strip RSs before passing to JSON parser).
It also causes unexpected behaviour from tools that handle text. For instance, the Chrome browser usually *displays* text files in the browser window, but if it has an RS character the file is downloaded instead!


RS could conceivably help detect *slightly* more cases of accidentally truncated writes (in the middle of a top-level number, in a top-level string in the middle of an escape sequence). That is not compelling. Particularly as it still fails to detect truncation of entire records (including RSs) so it is useless as a proper integrity mechanism. Finally, the "RS <entry> RS NL" suggestion misuses the normal semantics of a separator by using 2 per entry instead of 1.

-1 to RSs

--
James Manger