[OPSAWG] comments/edits on pcapng
Michael Richardson <mcr+ietf@sandelman.ca> Wed, 11 November 2015 01:05 UTC
Return-Path: <mcr@sandelman.ca>
X-Original-To: opsawg@ietfa.amsl.com
Delivered-To: opsawg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C4D0D1ACE5A for <opsawg@ietfa.amsl.com>; Tue, 10 Nov 2015 17:05:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.911
X-Spam-Level:
X-Spam-Status: No, score=-1.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id opc6lkEuKo-Q for <opsawg@ietfa.amsl.com>; Tue, 10 Nov 2015 17:05:27 -0800 (PST)
Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 00AF41ACE5B for <opsawg@ietf.org>; Tue, 10 Nov 2015 17:05:26 -0800 (PST)
Received: from sandelman.ca (obiwan.sandelman.ca [209.87.249.21]) by tuna.sandelman.ca (Postfix) with ESMTP id E4F32200A5; Tue, 10 Nov 2015 20:09:22 -0500 (EST)
Received: by sandelman.ca (Postfix, from userid 179) id C310E637F8; Tue, 10 Nov 2015 20:05:25 -0500 (EST)
Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id A57C8637F3; Tue, 10 Nov 2015 20:05:25 -0500 (EST)
From: Michael Richardson <mcr+ietf@sandelman.ca>
To: opsawg@ietf.org, Pcap-ng file format <pcap-ng-format@winpcap.org>
X-Attribution: mcr
X-Mailer: MH-E 8.6; nmh 1.3-dev; GNU Emacs 24.4.2
X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m<nPbLgmtKK-5dC@#:k
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg="pgp-sha1"; protocol="application/pgp-signature"
Date: Tue, 10 Nov 2015 20:05:25 -0500
Message-ID: <28578.1447203925@sandelman.ca>
Sender: mcr@sandelman.ca
Archived-At: <http://mailarchive.ietf.org/arch/msg/opsawg/yJ7LczDObabWAH4SrmJ886bqMVg>
Cc: dominique barthel <dominique.barthel@orange-ftgroup.com>, tcpdump-workers <tcpdump-workers@lists.tcpdump.org>
Subject: [OPSAWG] comments/edits on pcapng
X-BeenThere: opsawg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: OPSA Working Group Mail List <opsawg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/opsawg>, <mailto:opsawg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/opsawg/>
List-Post: <mailto:opsawg@ietf.org>
List-Help: <mailto:opsawg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/opsawg>, <mailto:opsawg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Nov 2015 01:05:29 -0000
11hr trips home are good for document review... Is the XML around? Found it: https://github.com/pcapng/pcapng/pull/38 I reviewed draft-tuexen-opsawg-pcapng, while it hasn't had an -01, some work has been occuring in the github copy, and so the effort isn't dead, just not as visible. For those that don't know, I'm been the packager/maintainer for tcpdump and libpcap for about the last 15 years, and until 5-6 years ago, pretty much anyone that used *pcap* format, used libpcap to read/write it. That has changed (which is good!) since the advent of the pcapng format. I personally didn't have a lot of time to review the ideas when they were in their formative stage. Aside from my initial whine, I think the document is quite mature, and I'd like to see it move forward, either within opsawg, or as ISE. Tcpdump has been able to read pcapng files for awhile now, but still writes pcap format files. This is definitely not my first read of the document, but the idea of going via OPSAWG surfaced at IETF90 (Toronto), but seems to have gotten stuck? In general, I had thought that pcapng was going to start with an identical magic as pcap, but that the "version_major" would say, "2", i.e: pcap1: bpf_u_int32 magic = 0xa1b2c3d4 (mutated for byte order) u_short version_major; u_short version_minor; when in fact, pcapng files start with: int32_t type = 0x0A0D0D0A int32_t length; int32_t byte_order_magic; int16_t major; int16_t minor; and the initial block isn't any different than any other blocks. I see the logic for this, and I don't have a problem with this on account of working, deployed code, I just wish they could all have been "pcap" files. So this is more a whine than a comment. I also find "pcapng" a poor long-term name, mostly it's the "ng" part that I dislike. .pc2p or something might have been better. section 4.1 says this about major/minor: Major Version: number of the current mayor version of the format. Current value is 1. This value should change if the format changes in such a way that code that reads the new format could not read the old format (i.e., code to read both formats would have to check the version number and use different code paths for the two formats) and code that reads the old format could not read the new format. Minor Version: number of the current minor version of the format. Current value is 0. This value should change if the format changes in such a way that code that reads the new format could read the old format without checking the version number but code that reads the old format could not read all files in the new format. and I think that I don't buy it. [First, "major version" should have been started at "2" :-) ] But, more importantly, we have an easily, and almost always incremenally extendable file format. It's hard to imagine a change so drastic that it would be worth incrementing the major version number that would still leave enough of the file format intact that there would be any value left. One would also wind up changing magic/header numbers. With protocols that are online, it is different, as there can be a bit of negotiation when major numbers don't match, but not really the case for files resting on disk. So I think that we should simply turn this into a 4-byte int32 field. Let's just put the highest *RFC* number that must be understood in order to comprehend all the blocks present. At approx. 3500 RFCs/10 years, 32-bits will last a long long long time... We could even do this with just the Minor Version and be good until rfc65535, which might be by oh year 2265 assuming the IETF continues at it's present rate. Maybe we could have a bit at the top which says, "There are experimental blocks, and understanding them is critical to the semantic content of file" On the Interface Description Block, (section: 4.2), the table shown should be an IANA controlled registry. The reference to section 3.5 is initially confusing when reading. I think that there should be a single table (in section 3.5). I see that the Section Header Block also reuses codes used in the IDB: I think that this is a mistake, the codes in a comment block should not be context dependant. My pull request takes care of all of them, I hope. I numbered them sequentially, but we could leave gaps. We will need a registration formula for this registry, and I think that First-Come / First Served (RFC5226/BCP26, section 4.1) will do fine for this one. On the subject of the Encryption Block. In discussion on the list many asked if we needed it. Let me suggest that we do, but that yes, we need more information in it. First, we need to make it look more like ESP (RFC4303): +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ---- | Security Parameters Index (SPI) | ^Int. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov- | Sequence Number | |ered +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ---- | Payload Data* (variable) | | ^ ~ ~ | | | | |Conf. + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov- | | Padding (0-255 bytes) | |ered* +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | | Pad Length | Next Header | v v +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ------ | Integrity Check Value-ICV (variable) | ~ ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ But, we don't need a SeqNumber, or a Next Header. We can probably assume that the things inside are already padded to 32-bit boundary, but self-describing padding is easy and simple. Yes, we should have an ICV. I make the SPI 64-bits to give it more space. Who can decrypt and even with what algorithm is actually unimportant/private. If you have to ask, then it's not you! Further a comment in the SHR would permit proper document management, if the file was part of an evidence locker. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Security Parameters Index (SPI -- 64 bits) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Data* (variable) | ~ ~ | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | Padding (0-255 bytes) | +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+ | | Pad Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Integrity Check Value-ICV (variable) | ~ ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Now, about LinkType: I suggest that having libpcap/tcpdump continue to maintain the table is not ideal. Instead, let's create an IANA registry for it, populate it with existing values, and make the libpcap group the Expert Review. -- Michael Richardson <mcr+IETF@sandelman.ca>, Sandelman Software Works -= IPv6 IoT consulting =-
- [OPSAWG] comments/edits on pcapng Michael Richardson
- Re: [OPSAWG] [pcap-ng-format] comments/edits on p… Guy Harris
- Re: [OPSAWG] comments/edits on pcapng Warren Kumari
- Re: [OPSAWG] [tcpdump-workers] [pcap-ng-format] c… Michael Richardson
- Re: [OPSAWG] comments/edits on pcapng Michael Richardson
- Re: [OPSAWG] [tcpdump-workers] [pcap-ng-format] c… Guy Harris