[dispatch] Introduction to qlog

Robin MARX <robin.marx@uhasselt.be> Thu, 25 February 2021 16:17 UTC

Return-Path: <robin.marx@uhasselt.be>
X-Original-To: dispatch@ietfa.amsl.com
Delivered-To: dispatch@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 1C1363A1D36 for <dispatch@ietfa.amsl.com>; Thu, 25 Feb 2021 08:17:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=uhasselt.be
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 4YGrQmt5YrNU for <dispatch@ietfa.amsl.com>; Thu, 25 Feb 2021 08:17:38 -0800 (PST)
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 750FC3A1D07 for <dispatch@ietf.org>; Thu, 25 Feb 2021 08:17:23 -0800 (PST)
Received: by mail-wr1-x42e.google.com with SMTP id r3so5800442wro.9 for <dispatch@ietf.org>; Thu, 25 Feb 2021 08:17:23 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uhasselt.be; s=google; h=mime-version:from:date:message-id:subject:to; bh=q05YeCu0npdmdfoCb06sre4gsWBtXEFoyv9murHplQA=; b=Qam3F2SZp5RvM+0Oc6hnSvt6D7LIOKQyD5xSNTrSQDP6myD8Y3likNfajf1b/HT2SQ oskdWusrSqXnsoAHvvc+Y7XZXfOnJKMTu5trq15CThzQvn1bQ022nj7w3it4rVeCYkUY ZIDcWQiWUt9/rrK75Xmdx0CYQtDT0ViV62yak=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=q05YeCu0npdmdfoCb06sre4gsWBtXEFoyv9murHplQA=; b=hob+X83M1BBLCZOtnBWXS1DHiI80UfL14o6VV+TkI5tsOLJhX5VI0IgYQetNix/QPw Fj5pIOCZrGSjzwafGkBssYHoOamaMXHfTKKyE+30TlBZcIRRc58yMhHyWOFJ9pJnCzNz aaJ5h7DKYkcwyxkjpBGdiWNBz2jaCvR2G1VSFmPUL90icLnrRj9TM9Zyaqsmp25lsUr1 wxYm2/xZ+6TItP7ygr9yDVP7rwwgqEyUehhoAl7Bu2IJBLktd+KC7bx33NVsDG2i+0rC EkQ32LhsmZW2Zy78HdSFGb8amtsf964VsV/PE7PnyP3DFAhSfjdWaNVm0zhYflwE2R7K rCQA==
X-Gm-Message-State: AOAM533PaBQd8zDkFhNqaNbKLCF2gRPW0Imt3XldzeV+sDF7vl6kp1Bn 5qWzwUk+Y6eVhl+lqOZ9TQaL2PTa4WBW3iMZGiG7fNdYyXY/Yg==
X-Google-Smtp-Source: ABdhPJyrhZMrS2SzheW11L9yURyGQHqeqT54XCdNOt46lAXPYOCJ81uAicvEecLgvxSdYqEXFHVduoeqNyKcTJB/Qk0=
X-Received: by 2002:adf:f750:: with SMTP id z16mr4393389wrp.108.1614269842213; Thu, 25 Feb 2021 08:17:22 -0800 (PST)
MIME-Version: 1.0
From: Robin MARX <robin.marx@uhasselt.be>
Date: Thu, 25 Feb 2021 17:17:10 +0100
Message-ID: <CAC7UV9Yn2d35w_SPYE09ZZDePLBmnOfnctw2Oa551ZP7Nz5EJg@mail.gmail.com>
To: dispatch@ietf.org
Content-Type: multipart/alternative; boundary="00000000000084633c05bc2b7c54"
Archived-At: <https://mailarchive.ietf.org/arch/msg/dispatch/oGFtO7Zypgt_EgpzMu0ZOr7tpUE>
Subject: [dispatch] Introduction to qlog
X-BeenThere: dispatch@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DISPATCH Working Group Mail List <dispatch.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dispatch>, <mailto:dispatch-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dispatch/>
List-Post: <mailto:dispatch@ietf.org>
List-Help: <mailto:dispatch-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dispatch>, <mailto:dispatch-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 25 Feb 2021 16:17:47 -0000

Hello Dispatch,

In two weeks I have a slot during the meeting to talk with you about the
qlog project.
Since this is the first time this is discussed in this wg, your chairs
asked me to do a small introduction via the mailing list in preparation.

qlog [1][2] started off as a way to do logging for HTTP/3 and QUIC (hence
Quic LOGging).
As QUIC encrypts almost all of its metadata, utilizing packet captures for
analysis almost always requires full decryption of application (user) data
as well, leading to potential scalability and especially privacy issues.

As such, qlog instead proposes logging protocol metadata at the
"endpoints"/implementations directly (e.g., client, server, load balancer,
...), where only the necessary (and properly anonymized) metadata can be
This approach additionally allows the inclusion of events typically not
seen on the wire, such as congestion control behaviour.
All events are recorded in a structured format (currently JSON) using a
fixed schema to make it easier to write cross-implementation tooling.

This approach has since found some success for QUIC and HTTP/3, with the
majority of implementations supporting the format [3] (or something
similar) and actively using its associated qvis tooling [4] to debug and
analyse implementations and deployments.
As such, the qlog drafts are on track to be adopted by the QUIC wg
following their re-charter after delivering QUIC v1 in the coming months.

However, it is clear that qlog's basic principles (mainly: structured
logging at endpoints) can be useful for many other (encrypted) protocols
besides QUIC and HTTP/3 as well.
As such, while for practical reasons the continued qlog work will happen in
the QUIC wg, the goal is to define it as a protocol-agnostic framework,
complete with guidelines to add event definitions for new protocols.
This can already be seen in the current split in two drafts: the first
defines a general-purpose schema with the format and high-level metadata
[1], while the QUIC and HTTP/3-specific events are in the second document
The idea would be to have different documents for additional protocols
added in the future.

In order to make sure qlog can indeed eventually be used as a substrate for
many different protocols and use cases, we are now already
soliciting feedback and insights from the wider IETF community.
My presentation on qlog in two weeks will give a bit more details on qlog,
how it has been used in practice and about the main open challenges we hope
you can help us with.
It will hopefully also entice some of you to join the later discussions in
the QUIC wg as well, of course ;)

See you all in two weeks!
With best regards,

[1]: https://datatracker.ietf.org/doc/draft-marx-qlog-main-schema/
[4]: https://qvis.quictools.info


dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

T +32(0)11 26 84 79 - GSM +32(0)497 72 86 94

Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05