Re: [Ext] Consensus call for qlog serialization format (issue #144)

[Lucas Pardue <[e-mail]>]

Hi Paul,


On Thu, Aug 5, 2021 at 5:41 PM Paul Hoffman <[e-mail]> wrote:

>
>
> We are having a disconnect here that is central to the question in this
> consensus call. The original call said:
>
> > The feeling in the room was to keep the JSON serialization format.
> Noting that implementations can use their own intermediate formats and
> transform to and from JSON as needed, and that future documents could
> specify other interop formats if there is sufficient interest.
> >
> > The proposed resolution for this matter is to keep the JSON
> serialization format as the canonical interoperable format. This email
> seeks to establish consensus for this. If you have comments, for or
> against, please respond on the issue. The call will run until EoD August 9
> 2021.
> >
>
>
> The call asked whether the (now "a") JSON serialization format will be the
> canonical interoperable format. That is quite different than "including at
> least one serialization format". As a concrete example:
>
> - The JSON serialization says that numbers such as packet_number may be
> represented as a number or a string
>
> - The data format says that the numbers such as packet_number are uint64
>

> If the JSON serialization format is the canonical interoperable format,
> and I'm writing a CBOR emitter, I would be allowed to write packet_number
> as a number or string because both could convert to JSON. However, if the
> data format is the canonical interoperable format, I would only be able to
> write it as a number.
>
> Thus, it is critical for interoperability between formats to specify if
> the data definition is canonical, or if the JSON serialization is
> canonical. If the latter, there really is no need for the data definitions
> at all; if this choice is made, interoperability would be more likely if
> the data definitions were removed.
>
> I hope this helps clarify why the WG needs to choose one or the other.
>

Thanks for the additional commentary. I fear my use of canonical hasn't
helped clarity much (blame my co-chair for allowing me to use big words
:-)).

The adopted qlog documents included a TypeScript-based data model (schema)
and a JSON serialization. There have been some discussions about changing
either of these. As you mentioned upthread, the two are associated. In an
attempt to keep discussions focused, we've somewhat avoided mentioning the
schema. The aim of finding consensus on the serialization format was so
that we could use that as input into deciding on schema definition. Picking
JSON for serialization would, hypothetically speaking for now, allow us to
rewrite the schema in CDDL. Using an IETF format for schema definition
would assuage some of the concerns that have been expressed about
Typescript or other options. And I anticipate a schema rewrite would
require the WG tackle some of the thornier issues in good time. Speaking
personally, I think that nailing down JSON as the working format du jour
will allow us to tackle data definition aspects that have been hanging over
the spec for a while.

qlog has an objective to support alternative serializations formats and
robust transformations between them. If I understand your comments
correctly. There's an argument to say we might be approaching things
backwards and that the group should pick a data definition language first,
and then the serialization stuff might just come out in the wash. It's a
bit chicken and egg. Given we have interoperability today between QUIC
implementations generating qlogs in JSON and tools consuming JSON, for many
people the decision about data definition is of less immediate practical
importance. But that's not to downplay the long term importance of creating
a robust data definition.

Speaking as a co-chair, getting clear consensus about the topic of schema
and serialization is important at this stage of the document's lifecycle. I
want to make sure we're asking the right questions now, and we understand
the potential impacts of things we might be trying to defer. So thanks for
picking at this. Does the intent to revist (soon) the qlog schema and
(possibly) extract the serialization address some of your concerns? Is
there a clearer way we could articulate the question to ensure the WG
understands what it is agreeing to?

Cheers,
Lucas