[dispatch] Introduction to qlog

Robin MARX <robin.marx@uhasselt.be> Thu, 25 February 2021 16:17 UTC

MIME-Version: 1.0
From: Robin MARX <robin.marx@uhasselt.be>
Date: Thu, 25 Feb 2021 17:17:10 +0100
Message-ID: <CAC7UV9Yn2d35w_SPYE09ZZDePLBmnOfnctw2Oa551ZP7Nz5EJg@mail.gmail.com>
To: dispatch@ietf.org
Content-Type: multipart/alternative; boundary="00000000000084633c05bc2b7c54"
Archived-At: <https://mailarchive.ietf.org/arch/msg/dispatch/oGFtO7Zypgt_EgpzMu0ZOr7tpUE>
Subject: [dispatch] Introduction to qlog
Precedence: list

Hello Dispatch,

In two weeks I have a slot during the meeting to talk with you about the
qlog project.
Since this is the first time this is discussed in this wg, your chairs
asked me to do a small introduction via the mailing list in preparation.

qlog [1][2] started off as a way to do logging for HTTP/3 and QUIC (hence
Quic LOGging).
As QUIC encrypts almost all of its metadata, utilizing packet captures for
analysis almost always requires full decryption of application (user) data
as well, leading to potential scalability and especially privacy issues.

As such, qlog instead proposes logging protocol metadata at the
"endpoints"/implementations directly (e.g., client, server, load balancer,
...), where only the necessary (and properly anonymized) metadata can be
recorded.
This approach additionally allows the inclusion of events typically not
seen on the wire, such as congestion control behaviour.
All events are recorded in a structured format (currently JSON) using a
fixed schema to make it easier to write cross-implementation tooling.

This approach has since found some success for QUIC and HTTP/3, with the
majority of implementations supporting the format [3] (or something
similar) and actively using its associated qvis tooling [4] to debug and
analyse implementations and deployments.
As such, the qlog drafts are on track to be adopted by the QUIC wg
following their re-charter after delivering QUIC v1 in the coming months.

However, it is clear that qlog's basic principles (mainly: structured
logging at endpoints) can be useful for many other (encrypted) protocols
besides QUIC and HTTP/3 as well.
As such, while for practical reasons the continued qlog work will happen in
the QUIC wg, the goal is to define it as a protocol-agnostic framework,
complete with guidelines to add event definitions for new protocols.
This can already be seen in the current split in two drafts: the first
defines a general-purpose schema with the format and high-level metadata
[1], while the QUIC and HTTP/3-specific events are in the second document
[2].
The idea would be to have different documents for additional protocols
added in the future.

In order to make sure qlog can indeed eventually be used as a substrate for
many different protocols and use cases, we are now already
soliciting feedback and insights from the wider IETF community.
My presentation on qlog in two weeks will give a bit more details on qlog,
how it has been used in practice and about the main open challenges we hope
you can help us with.
It will hopefully also entice some of you to join the later discussions in
the QUIC wg as well, of course ;)

See you all in two weeks!
With best regards,
Robin

[1]: https://datatracker.ietf.org/doc/draft-marx-qlog-main-schema/
[2]:
https://datatracker.ietf.org/doc/draft-marx-qlog-event-definitions-quic-h3/
[3]:
https://qlog.edm.uhasselt.be/anrw/files/DebuggingQUICWithQlog_Marx_final_21jun2020.pdf
[4]: https://qvis.quictools.info

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

T +32(0)11 26 84 79 - GSM +32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

[dispatch] Introduction to qlog Robin MARX