Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt

Richard Gibson <> Mon, 03 July 2017 23:22 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 173471317BF for <>; Mon, 3 Jul 2017 16:22:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Thg24hDErl6m for <>; Mon, 3 Jul 2017 16:22:41 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:400c:c08::248]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 1E6A61317B2 for <>; Mon, 3 Jul 2017 16:22:36 -0700 (PDT)
Received: by with SMTP id j53so77215789uaa.2 for <>; Mon, 03 Jul 2017 16:22:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0B2zkxjxvfobK/XfBhGR4Qcj4Jg7qscXF8mXglUKcx0=; b=H+IQq4ujTnWbnC6R88PASySf3F9oLPB/G2VGmfPnuM96DbSAZvi+QlXm3nlGMqeQn+ j/a4vD1N5CTA8ty3irwpDTmvm0l5bbNRTR11ASHkbRqidOCX4GAo6P4kxepRwZn8hu0M KkaHTKnFwGO58ksF5Hh9boeD+qEA1gFSgW2DQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0B2zkxjxvfobK/XfBhGR4Qcj4Jg7qscXF8mXglUKcx0=; b=XIbV0RTJEwmPE/Q8AwAuvI+29a23vuN9hkFts3gO350PkbhvcHsa08pKKwXFJDVlO4 a7pcPAJJH0D/p0VcvGI+DhpQPOdtw7W+Dkne83XAUl/2BWH40MHnttyUUHv0RktOfSeB Ls4SAEhe1Pc8fibPx1+EujCA5ZSCNlhly9U5ObLqwCXCradSfAlmjsKN4LlwlGp09lEg EkmHNS9yPjuTspIk/XBr9+u96a0+rP5SoeDWiKcwbZTvCvAZtmC6V8y8Q+iKUcKbWrYu 5benZF1YKBcFOM+GqpJ4MX/A9OtoZ4am/2+vbD+vCwCwqW3pYZAmrOZQfuN6wh2B87HY wC+Q==
X-Gm-Message-State: AKS2vOzxoXLEWEnqASFzcDdJLNQiXbe+FeruYW9LVsPu0Bk2/VqBS4V2 KJKL7h2BeMQU3XnG9wN2TgjgMeg9/AWP
X-Received: by with SMTP id i18mr20289617uah.62.1499124155163; Mon, 03 Jul 2017 16:22:35 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Mon, 3 Jul 2017 16:22:14 -0700 (PDT)
In-Reply-To: <>
References: <>
From: Richard Gibson <>
Date: Mon, 3 Jul 2017 19:22:14 -0400
Message-ID: <>
Cc:, dnsop <>
Content-Type: multipart/alternative; boundary="f403043c449cbe3bf30553720a8f"
Archived-At: <>
Subject: Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IETF DNSOP WG mailing list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 03 Jul 2017 23:22:50 -0000

I looked over this draft in detail, and found a handful of ambiguous points
("Clarifications" and "Potentially Missing Data" below). But more
importantly, it is very close to defining a format that could replace much
of my organization's in-house technology. Would you consider some
generalizations to take it over the finish line ("Extension Fields" and
"Opt-in Lossyness")? Only the suggestions related to representing time and
"classtype" items would change the representation of existing data in such
a way that implementations already supporting the draft specification would
require changes.

* Items in the "classtype" table (section 7.11) are missing data type
documentation. Both "type" and "class" should be unsigned numbers.
  * And speaking of 7.11, why are CLASS/TYPE pairs represented as CBOR maps
instead of more efficient two-item arrays? If it was an intentional
decision for clarity, then maybe the section 7.7 block preamble
"earliest-time" field should also be promoted to a map ("time-seconds",
"time-useconds", "time-pseconds", mirroring Q/R items) for the same reason.
* In "query-sig" table items (section 7.13) "transport-flags" field, the
bit corresponding to "trailing bytes" shouldn't be limited to UDP.
* In section 7.18, "and an unsigned key" appears to be meaningless and
should probably be removed.

*Potentially Missing Data*
* In "query-sig" table items (section 7.13), "transport-flags" should
probably be extended to include a TLS bit (cf. RFC 7858).

*Extension Fields*
Of the many potentially open-ended key-value maps (file preamble, file
preamble configuration, block preamble, block statistics, query signatures,
Q/R data), only block statistics allows for "implementation-specific
fields", and no further guidance is provided. I think all maps should allow
such fields, with a recommendation that they use an implementation-specific
prefix to avoid collisions with fields added by other implementations or
later versions of C-DNS. Example use cases:
* Extend file preamble configuration (section 7.5) to document aggregation
of queries answered by wildcard names.
* Extend the block preamble (section 7.7) to override file preamble fields
like "host-id" and "server-addresses", enabling fleet-wide file merges.
* Extend query signatures (section 7.13) to indicate EDNS Client Subnet use
(cf. RFC 7871) or selection of context-dependent handling like views.
* Extend Q/R data (section 7.18) to include actual EDNS Client Subnet
values or even a "count" field (which would come with *tremendous* space
savings, sufficient for my organization to abandon its internal
summarization format in favor of C-DNS).

*Opt-in Lossyness*
The format is generally quite good about allowing for detail without
requiring it. However, there are some areas where more space savings could
be had:
* Communicate aggregation of IP addresses into prefixes (i.e., the
irrelevance of least-significant bits in ip-address values) with new
"client-prefix-length-ipv4" and "client-prefix-length-ipv6" and
"server-prefix-length-ipv4" and "server-prefix-length-ipv6" file preamble
configuration options.
* Communicate case-normalizing aggregation of names (e.g., transforming
"" into "") with a new boolean-valued
"name-normalization" file preamble configuration option.
* In "rr" table items (section 7.15), "ttl" should be optional to
accommodate decrementing in recursive resolver responses.
* In Q/R data items (section 7.18) and malformed packet records (section
7.20), I'd like "time-useconds" broken out into "time-seconds" and optional
"time-useconds", both for parity with block-preamble "earliest-time" and
for space savings in applications that are content with second-level
* For truly customizable aggregation, I think all query signature (section
7.13) and Q/R (section 7.18) data item fields should be optional... but
especially Q/R data "client-port" and "transaction-id".

On Mon, Jul 3, 2017 at 5:08 AM, <> wrote:

> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
> This draft is a work item of the Domain Name System Operations of the IETF.
>         Title           : C-DNS: A DNS Packet Capture Format
>         Authors         : John Dickinson
>                           Jim Hague
>                           Sara Dickinson
>                           Terry Manderson
>                           John Bond
>         Filename        : draft-ietf-dnsop-dns-capture-format-03.txt
>         Pages           : 49
>         Date            : 2017-07-03
> Abstract:
>    This document describes a data representation for collections of DNS
>    messages.  The format is designed for efficient storage and
>    transmission of large packet captures of DNS traffic; it attempts to
>    minimize the size of such packet capture files but retain the full
>    DNS message contents along with the most useful transport metadata.
>    It is intended to assist with the development of DNS traffic
>    monitoring applications.
> The IETF datatracker status page for this draft is:
> There are also htmlized versions available at:
> capture-format-03
> A diff from the previous version is available at:
> Please note that it may take a couple of minutes from the time of
> submission
> until the htmlized version and diff are available at
> Internet-Drafts are also available by anonymous FTP at:
> _______________________________________________
> DNSOP mailing list