[Qlog] qlog in pcap-ng format

Michael Richardson <mcr+ietf@sandelman.ca> Sat, 08 May 2021 19:43 UTC

Return-Path: <mcr+ietf@sandelman.ca>
X-Original-To: qlog@ietfa.amsl.com
Delivered-To: qlog@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A75583A0E27 for <qlog@ietfa.amsl.com>; Sat, 8 May 2021 12:43:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.199
X-Spam-Level:
X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lASRMLRUgGuQ for <qlog@ietfa.amsl.com>; Sat, 8 May 2021 12:43:02 -0700 (PDT)
Received: from tuna.sandelman.ca (tuna.sandelman.ca [209.87.249.19]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E85F03A0E26 for <qlog@ietf.org>; Sat, 8 May 2021 12:43:01 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by tuna.sandelman.ca (Postfix) with ESMTP id DE8B5183DC for <qlog@ietf.org>; Sat, 8 May 2021 15:51:46 -0400 (EDT)
Received: from tuna.sandelman.ca ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with LMTP id lMGKQsaT3FiW for <qlog@ietf.org>; Sat, 8 May 2021 15:51:45 -0400 (EDT)
Received: from sandelman.ca (unknown [IPv6:2607:f0b0:f:2:56b2:3ff:fe0b:d84]) by tuna.sandelman.ca (Postfix) with ESMTP id C841A18047 for <qlog@ietf.org>; Sat, 8 May 2021 15:51:45 -0400 (EDT)
Received: from localhost (localhost [IPv6:::1]) by sandelman.ca (Postfix) with ESMTP id D3FA72B3 for <qlog@ietf.org>; Sat, 8 May 2021 15:42:58 -0400 (EDT)
From: Michael Richardson <mcr+ietf@sandelman.ca>
To: qlog@ietf.org
X-Attribution: mcr
X-Mailer: MH-E 8.6+git; nmh 1.7+dev; GNU Emacs 26.1
X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m<nPbLgmtKK-5dC@#:k
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg="pgp-sha512"; protocol="application/pgp-signature"
Date: Sat, 08 May 2021 15:42:58 -0400
Message-ID: <32242.1620502978@localhost>
Archived-At: <https://mailarchive.ietf.org/arch/msg/qlog/pfChbNDjSKR2o5AP1v2f1A4AFVk>
Subject: [Qlog] qlog in pcap-ng format
X-BeenThere: qlog@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qlog.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/qlog>, <mailto:qlog-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/qlog/>
List-Post: <mailto:qlog@ietf.org>
List-Help: <mailto:qlog-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/qlog>, <mailto:qlog-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 08 May 2021 19:43:07 -0000

I saw part of this presentation a few weeks ago someplace.
I just watched the SAAG recording.

1) I think that JSON is too verbose for this stuff, but I think that CBOR
   (described by CDDL) is probably exactly right for what you are doing.

2) PCAP-NG, which wireshark reads/writes, and libpcap only reads,
   accomodates a multitude of different streams and formats, not just
   ethernet dumps.
   (We have USB captures, nflog captures, and stuff all sorts of
   instruments).

PCAP and PCAP-NG are candidates for adoption by the OPSAWG, but if
QUIC would like to adopt them, that's okay with me.

I think that there is a significant value here, particular when there may be
a need to capture both TCP, UDP and internal logic at the same time.
Imagine something doesn't work, because something is eating UDP packets with
a particular checksum (or ethernet CRC).  It's happened more than once.
Having the internal state that says, "I retransmitted foo", and then the
capture from end A, that shows the encrypted QUIC packet going out, and
then the capture from end B, which says, "I didn't get foo, I asked for it
again"
would be very useful to see.  That requires merging streams from both ends,
and interleaving actual captured traffic at the same time.
(And still doesn't require access to cleartext to debug)

--
Michael Richardson <mcr+IETF@sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide