Re: [secdir] secdir review of draft-ietf-json-text-sequence-11

[Carl Wallace <[e-mail]>]

Inline...

On 12/16/14, 12:48 PM, "Nico Williams" <[e-mail]> wrote:

>On Tue, Dec 16, 2014 at 12:20:08PM -0500, Carl Wallace wrote:
>> On 12/16/14, 11:32 AM, "Nico Williams" <[e-mail]> wrote:
>> >OK, that will be section 3 (security considerations text), something
>> >like:
>> >
>> >   Parsing and re-encoding a JSON text sequence need not produce the
>> >   same sequence of octets.  Do not rely on being able to reproduce the
>> >   same inputs to a cryptographic integrity protection function.
>> 
>> If supporting signing is not important to anyone, OK. This seems like a
>> significant sacrifice, especially when positioned against the benefit of
>> adding but not removing <LF>s.
>
>Supporting validation of signed sequences by first re-encoding the
>sequence is absolutely not a goal.

Re-encoding is not the point. The value parsed from the sequence has an
extra <LF> relative to what was passed in and would thus break a signature
generated on the original value. I don’t think there’s any lack of clarity
on this point. You already noted that in your view either the entire
sequence must be signed or individual elements signed with a wrapper.  I
should have written “if supporting detached signing is not important to
anyone, OK”.

>
>This is almost dicta for all encodings of any messages.
>
>We found out long ago that it doesn't work when the encoding does have a
>canonical form (and even when it does but it's just not used by the
>signer).
>
>> >Section 2.3 actually says "malformed" in its first sentence.  It
>> >mentions truncation only as an example of why a JSON text might be
>> >malformed, in the second sentent.
>> 
>> I am making a distinction between failure to parse a JSON text and
>>failure
>> to parse a JSON text sequence. I think the text only addresses the
>>former.
>
>The whole section is about JSON text parse errors not being fatal for
>sequence parsing.  I don't understand the objection.  Perhaps if you
>propose text I will?

I think given the lending of <LF>s to the JSON text, there is not such
thing as a JSON text sequence parsing error. You find an RS to terminate
an element or you run out of bytes and terminate the element - no failures
at the sequence parsing level (though there may be errors that percolate
to the JSON parser).

>
>> >><snip>
>> >> [extensive discussion of the LF elided]
>> 
>> How can a decoder know that <RS>123<LF><RS> was what the originator
>> intended and not something that was terminated by the text sequence
>> encoder? The originator may have intended <RS>1234<ws><LF><RS>. There
>> seems to be some assumption that the supplier of JSON text may fail to
>> self-delimit but would not fail to supply the full value. It’s a
>>contrived
>> example, but how should an incremental JSON parser handle texts returned
>> from a parser operating on the sequence: <RS>123<LF><RS>4<ws><LF><RS>?
>> Would it be two values 123 and 4 or one value 1234? Why is it not be
>> preferable to report an error here <RS>123<LF><RS> instead of trying to
>> auto-terminate it when encoding the sequence?

You have avoided this question in a couple of forms now. An answer here
would probably clarify things tremendously with regard to <LF> additions
and how incremental parsing or detecting incomplete encoding is supposed
to work.

>
>The assumption is that the "process" writing the sequence will properly
>encode the sequence elements,

My assumption was “a" process encoding a “sequence" may receive an
"encoded sequence element” from “another” process, possibly as an API call
from a different box.

>and will write the <RS><element><LF>
>sequence correctly.

The point is what if element is incomplete, either due to failure or
incremental contribution to the sequence?

><POSIX discussion elided>
> This too is out of scope.

Which is fine because it is not the point.

><snip>
>> 
>> I guess we just disagree on whether the text sequence encoder is
>> necessarily in a position to terminate data that may be incrementally
>> supplied or incompletely supplied by a caller and whether or not this
>> important function should be allocated to the caller instead of to the
>> JSON text sequence encoder.  One alternative would be to add a <ws> only
>
>Of course a properly functioning encoder on a properly function system
>is in a position to terminate each element.  How can this be in doubt?

The JSON text encoder may be on one system and the JSON text sequence
encoder may be on another system (one of the examples in the draft is for
logs, so this must already be assumed as possible). In the example above,
if the text encoder fails after sending 123, the text sequence encoder
will add an <LF> and a decoder will not detect truncation. How can this be
in doubt?

>
>A sequence encoder might write() RS, then invoke an incremental JSON
>text encoder to encode and write() the JSON text, then finally when the
>JSON text encoder completes its task, the sequence encoder write()s the
>LF.

This may be the source of confusion. Does incremental parsing encompass a
single text sequence element or multiple text sequence elements?  Or can
it be either way?

>
>The sequence encoder can also marshall the whole thing into a buffer and
>write() that.
>
>Part of the point of JSON text sequences is to permit online processing
>of large data sets without requiring a streaming JSON text parser (which
>is not the same as an incremental parser).  That point also applies to
>encoding: if the sequence elements are small enough to fit into memory
>and be parsed non-incrementally, then they also necessarily meet similar
>constraints on the encoder side.
>
>The only way to screw this up is when breaking down the process of
>writing a sequence into a non-atomic sequence of operations that race in
>a multi-process/threaded writer.  The details of that problem and how to
>avoid it are clearly out of scope for this document.
>
>> when a non-self-delimited text is passed to a non-incremental encoder,
>> then encode that altered value into the sequence (and terminate all
>> sequence elements with <RS> only).
>
>The RS/LF bracketing was the result of lengthy discussions on the JSON
>WG list.  The RS/LF bracketing design should not be reconsidered at this
>time unless you have a security concern that cannot be addressed
>otherwise.  I reject the concern about validation of signatures via the
>use of re-encoded sequences (see above and earlier)

I never associated re-encoded sequences with signature verification. I
only asked about <LF> accumulation during re-encoding.

>-- it is commonly
>accepted and strongly recommended practice that signatures should be
>validated over what is signed, then and only then (after the signature
>is validated successfully) should the payload be parsed.  If you have
>any other security concerns relating to the LF, let's hear them.

No additional security concerns relating to the LF other than what has
been stated (non-support for detached signatures, potential to alter
interpretation of elements in some circumstances). As noted above,
re-encoding is not and has not been the point re: signature verification.

>
>Nico
>--