qlog: new streaming serialization format
Robin MARX <robin.marx@uhasselt.be> Wed, 20 October 2021 14:19 UTC
Return-Path: <robin.marx@uhasselt.be>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EFC843A0970 for <quic@ietfa.amsl.com>; Wed, 20 Oct 2021 07:19:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=uhasselt.be
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TlUKYP9LjMOB for <quic@ietfa.amsl.com>; Wed, 20 Oct 2021 07:19:25 -0700 (PDT)
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 48F8C3A0898 for <quic@ietf.org>; Wed, 20 Oct 2021 07:19:24 -0700 (PDT)
Received: by mail-wm1-x334.google.com with SMTP id 77-20020a1c0450000000b0030da2439b21so2203473wme.0 for <quic@ietf.org>; Wed, 20 Oct 2021 07:19:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uhasselt.be; s=google; h=mime-version:from:date:message-id:subject:to; bh=NtNjqCGMwOScsllQDg8/hWz/sQK7lsEqPXvElFt8ORM=; b=AsyyBtda0ho9wV9QPuw1mzHpDOiKD0GjQni177NA4IRmvW1IRoMVaMQGI2rGMh8s/7 WkfvOn6gPydcnVhn/ADLnZ52HT0xgCFOSWpZlKFxlixBzBBLVmqYGavXttjUfKtuosjg rRfse9ZCQuMA9EjDuPkmcKuVMLzQggCvdSL3E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=NtNjqCGMwOScsllQDg8/hWz/sQK7lsEqPXvElFt8ORM=; b=NeeWQkcF6N7xW4FE2zklYIsvlSDHJ+7zhK9Eq8Cb40pbDrEOcvSNxq6Vrd4tr/IUkv czOLfwcWLT/93RhXv9Oj8hFOTrrkY3mBdFp82qQjvvY6lAYKIEgbrmruVvEtNxOeQTlS OE0LjCstTmZI0yteEtALlErnvO9tDFIebFpqgxRkD32zIRty2LiqiK/RdVAFjSR4tLIj n/4NTX4BYSovimz3BDWVSY1KhZWQ0ufcN4iBLpnBrDLc4+t3PDJ5uk5iMprCQeZggfdr 5+stmnvOUMrvYpsEcbA54BF7BnA4WQtbVXLzO76XntSP0VwFXvoTByMP5+c5yKSvo+pW 9Hhw==
X-Gm-Message-State: AOAM5320YtC/et0e4uW6HWOnzDxalR3P8VJZgwZFoz0uCSBjSKX8+Vn3 kVDT+vLX1j5o9bP8N7aBLxreTUaBQihuM1hWWzGAK5rqe/8=
X-Google-Smtp-Source: ABdhPJzK/1yuXlK9HZIflBw4IlAeMZLSbgfGktTlW1DHvXH5HD+6U84tzMR94pix5esMvP5j8C/Mix6iizV8FcMLezM=
X-Received: by 2002:a05:6000:1563:: with SMTP id 3mr66954wrz.20.1634739560622; Wed, 20 Oct 2021 07:19:20 -0700 (PDT)
MIME-Version: 1.0
From: Robin MARX <robin.marx@uhasselt.be>
Date: Wed, 20 Oct 2021 16:19:16 +0200
Message-ID: <CAC7UV9ZqyDKAVctj6Q=NfAG=-sXvBJ+2fwScrWd8qhkUkKjPLw@mail.gmail.com>
Subject: qlog: new streaming serialization format
To: IETF QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000cfc14205cec97606"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/ikWj1UHov7Lbp1kO6TjlyChWIWo>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Oct 2021 14:19:31 -0000
Hello everyone, As discussed during IETF 111, we are looking to update qlog to move to a new streaming serialization format. The current version uses the "Newline Delimited JSON" (NDJSON) format for event streaming, which, while it works well, is not an official standard (in the IETF or elsewhere). As such, it was proposed we move to the "JSON text-sequences" format defined in RFC 7464 [1]. json-seq separates each record (qlog event in our case) with the Record Separator character (\x1E) and a \n at the end. As such, it should be easy to update an existing NDJSON implementation (as it just requires outputting one additional character per event). We have prototyped and tested this approach in qvis and various other tools like jq and the early outcomes are looking good [2]. As such, we will be preparing a PR and associated draft-01 for IETF 112 that introduces this change. In the meantime, it would be great if people interested in event streaming or with existing NDJSON qlog stacks could look at the proposal/early results [2] and provide feedback. [1]: https://datatracker.ietf.org/doc/html/rfc7464 [2]: https://github.com/quicwg/qlog/issues/172 With best regards, the qlog editors P.S. note that we also still plan to keep the normal JSON non-streaming format intact, leading to two "main" formats defined in the spec: 1) normal JSON and 2) JSON text sequences. -- dr. Robin Marx Postdoc researcher - Web protocols Expertise centre for Digital Media *Cellphone *+32(0)497 72 86 94 www.uhasselt.be Universiteit Hasselt - Campus Diepenbeek Agoralaan Gebouw D - B-3590 Diepenbeek Kantoor EDM-2.05
- qlog: new streaming serialization format Robin MARX
- Re: qlog: new streaming serialization format Robin MARX