Re: qlog: new streaming serialization format

[Robin MARX <[e-mail]>]

Hello all,

Thanks to everyone who provided feedback on the github issue.

We have now released a new draft of the qlog main schema detailing the new
streaming format [1].
qvis [2] has also been updated to support the new approach, so you can load
files using the new format/extension.
If you encounter any issues, please let me know.

I will be available during the hackathon next week for people who want to
implement this new setup and have questions/find bugs/etc.

I am also open to discuss qlog/qvis support for other emerging use cases,
such as multipath, masque, webtransport, etc. if people are interested.


[1]: https://www.ietf.org/id/draft-ietf-quic-qlog-main-schema-01.html
[2]: https://qvis.quictools.info/

With best regards,
Robin


On Wed, 20 Oct 2021 at 16:19, Robin MARX <[e-mail]> wrote:

> Hello everyone,
>
> As discussed during IETF 111, we are looking to update qlog to move to a
> new streaming serialization format.
>
> The current version uses the "Newline Delimited JSON" (NDJSON) format for
> event streaming, which, while it works well, is not an official standard
> (in the IETF or elsewhere).
>
> As such, it was proposed we move to the "JSON text-sequences" format
> defined in RFC 7464 [1]. json-seq separates each record (qlog event in our
> case) with the Record Separator character (\x1E) and a \n at the end. As
> such, it should be easy to update an existing NDJSON implementation (as it
> just requires outputting one additional character per event).
>
> We have prototyped and tested this approach in qvis and various other
> tools like jq and the early outcomes are looking good [2].
>
> As such, we will be preparing a PR and associated draft-01 for IETF 112
> that introduces this change.
>
> In the meantime, it would be great if people interested in event streaming
> or with existing NDJSON qlog stacks could look at the proposal/early
> results [2] and provide feedback.
>
> [1]: https://datatracker.ietf.org/doc/html/rfc7464
> [2]: https://github.com/quicwg/qlog/issues/172
>
>
> With best regards,
> the qlog editors
>
> P.S. note that we also still plan to keep the normal JSON non-streaming
> format intact, leading to two "main" formats defined in the spec: 1) normal
> JSON and 2) JSON text sequences.
>
> --
>
> dr. Robin Marx
> Postdoc researcher - Web protocols
> Expertise centre for Digital Media
>
> *Cellphone *[phone]
>
> www.uhasselt.be
> Universiteit Hasselt - Campus Diepenbeek
> Agoralaan Gebouw D - B-3590 Diepenbeek
> Kantoor EDM-2.05
>
>
>

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

*Cellphone *[phone]

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05