Re: [Qlog] qlog in pcap-ng format

[Robin MARX <[e-mail]>]

Hello Michael,

Thank you for your comments.

1)
The qlog drafts are set to be adopted by the QUIC wg soon and this is
indeed still an open point of discussion.
This is tracked at https://github.com/quiclog/internet-drafts/issues/144
with already some thoughts on the matter mentioned there.

2)
I agree there is significant overlap between PCAP and qlog conceptually
(though I did not know pcap was actually considered for adoption at the
IETF).
However, from my understanding, the PCAP format is strongly oriented around
logging (full) raw packet contents with little support for omission of
fields / anonymization besides rough, high-level truncation (-s, snaplen?)
As such, while it would be possible to support
*encrypted*-quic-with-extra-comments,
it would be much more difficult to support *decrypted*-quic-with-extra-comments
taking into account privacy/file size/ease of capture/...
(I'm not even sure pcap allows the logging of decrypted protocol data
as-is? And even if it does, is it still compatible with e.g., wireshark or
other pcap tools?)

Additionally, again from my understanding, pcap is a custom binary format
that's not all that easy to process/use.
IIUC, even tools like pyshark utilize tshark's XML output instead of
parsing the PCAP itself  (which has been a source of much frustration on my
end...).
This seems to go somewhat against one of the main goals of qlog, which is
to make the creation of tools and the analysis of data easier (which is one
of the main reasons to choose JSON in the first place).

That being said, I'm far from an expert on PCAP/PCAP-NG and could be wrong
on all these counts. Please feel free to educate me.
However, if I'm not, I feel it's still best to continue qlog as a separate,
complementary approach to pcaps.


Finally, I've added the QUIC wg in CC, as that's where most of the
work/discussion will likely be done in the future.

With best regards,
Robin


On Sat, 8 May 2021 at 21:43, Michael Richardson <[e-mail]>
wrote:

>
> I saw part of this presentation a few weeks ago someplace.
> I just watched the SAAG recording.
>
> 1) I think that JSON is too verbose for this stuff, but I think that CBOR
>    (described by CDDL) is probably exactly right for what you are doing.
>
> 2) PCAP-NG, which wireshark reads/writes, and libpcap only reads,
>    accomodates a multitude of different streams and formats, not just
>    ethernet dumps.
>    (We have USB captures, nflog captures, and stuff all sorts of
>    instruments).
>
> PCAP and PCAP-NG are candidates for adoption by the OPSAWG, but if
> QUIC would like to adopt them, that's okay with me.
>
> I think that there is a significant value here, particular when there may
> be
> a need to capture both TCP, UDP and internal logic at the same time.
> Imagine something doesn't work, because something is eating UDP packets
> with
> a particular checksum (or ethernet CRC).  It's happened more than once.
> Having the internal state that says, "I retransmitted foo", and then the
> capture from end A, that shows the encrypted QUIC packet going out, and
> then the capture from end B, which says, "I didn't get foo, I asked for it
> again"
> would be very useful to see.  That requires merging streams from both ends,
> and interleaving actual captured traffic at the same time.
> (And still doesn't require access to cleartext to debug)
>
> --
> Michael Richardson <[e-mail]>   . o O ( IPv6 IøT consulting )
>            Sandelman Software Works Inc, Ottawa and Worldwide
> --
> Qlog mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/qlog
>


-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

*Cellphone *[phone]

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05