Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-05.txt

Richard Gibson <> Wed, 28 February 2018 06:40 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id A220B12DA26 for <>; Tue, 27 Feb 2018 22:40:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.03
X-Spam-Status: No, score=-2.03 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 0ZNMYkRqM0Ly for <>; Tue, 27 Feb 2018 22:40:14 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 8E8FC12DA14 for <>; Tue, 27 Feb 2018 22:40:14 -0800 (PST)
Received: from pps.filterd ( []) by ( with SMTP id w1S6e69O114701; Wed, 28 Feb 2018 06:40:08 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=subject : to : references : from : message-id : date : mime-version : in-reply-to : content-type; s=corp-2017-10-26; bh=PZpY9W30aSacXl8WILbahHZjDw+jJtz6rHk7AGXlXVQ=; b=n2+kmBQyEP8Rccap21skua4p9VrwvOCkVetakVySTs1zUmgGUdrufHeC+3AMOVIC+D92 cy/lQ4nu8hyKbyppL8fy0lRAQLK173kJonq/jUm3DD8e3Cucj7klFS9x4SJnoGJYw86f d//VYLonjmYjItRgR//cg02chqLmvqb1xn8lIOWKDwXPJHcgZjg4d6GfQe2lNr1mrP50 5rgF7lUyksg/7/98CsQtVeYJyk9/gCTOcTocws4chJHnqWyADNyHv170nF79xFfHVwCg DhHe+6XeeGorVEXAsoEIvY/7PPiKls0x538UPtOoIGnoI/rGlVGUV257qHyXivRs9li8 Yw==
Received: from ( []) by with ESMTP id 2gdpq106xc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Feb 2018 06:40:06 +0000
Received: from ( []) by (8.14.4/8.14.4) with ESMTP id w1S6aqCZ025206 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 28 Feb 2018 06:36:52 GMT
Received: from ( []) by (8.14.4/8.14.4) with ESMTP id w1S6apt2005745; Wed, 28 Feb 2018 06:36:51 GMT
Received: from [] (/ by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 27 Feb 2018 22:36:51 -0800
To: Sara Dickinson <>, IETF DNSOP WG <>
References: <> <>
From: Richard Gibson <>
Message-ID: <>
Date: Wed, 28 Feb 2018 01:36:50 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: multipart/alternative; boundary="------------A7162A7582D3ADECF5949D7E"
Content-Language: en-US
X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8817 signatures=668681
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1802280078
Archived-At: <>
Subject: Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-05.txt
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IETF DNSOP WG mailing list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 28 Feb 2018 06:40:18 -0000

Thank you so much for the changes, I am confident that this format is 
now in a state where my organization could build on top of it to suit 
our needs. I have a handful of remaining comments below, mostly 
describing ways in which that process could be made more straightforward 
and standardized.

* Section 6.2.1 and At minimum, RR field "ttl" should be 
optional in order to accommodate aggregation of resolver responses in 
which that field is decremented over time. But probably "rdata-index" 
should be optional as well.

* Section 7.1: Negative integer map keys are concise, but they come with 
the potential for interoperability-hostile collisions. I think the 
document should encourage restricting their use to tightly-controlled 
closed systems, and then additionally propose a particular namespacing 
pattern (e.g., "{namespaceName}-{keyName}") for use of 
implementation-specific /string/ keys in other cases. Further, it should 
explicitly reserve positive integer keys for future standardized 
extensions to the C-DNS format.

* Section 7.4.1: Why is BlockParameters an array instead of a map?

* Section StorageParameters fields "opcodes" and "rrtypes" 
could be defined better... are they values /recorded/ (i.e., that were 
actually observed) or values that are /recordable/ (i.e., that the 
collection application was looking for)? And shouldn't they be optional 
(in either case, but *especially* the latter, lest implementations write 
out every possible rrtype value).

* Section 7.5.3: It's very good that the BlockTables "ip-address" items 
can be truncated by prefix length, but we have actually run into cases 
that call for /variable/ truncation (i.e., more precision in some 
subnets than in others), especially true given that a single table holds 
both client and server address data. What would you think about 
representing addresses not as direct byte strings, but rather as 
variable-length arrays or maps—index 0 encoding the prefix as a byte 
string, and optional index 1 (when present) encoding a prefix-length 
override? Or, if the extra size per address is too much, at least 
allowing such representations via polymorphism (byte string for default 
prefix-length, array/map for non-default)?

* Section QueryResponseSignature field "qr-transport-flags" 
seems to reserve 1 bit /each/ for UDP, TCP, TLS, and DTLS, but it seems 
to me that UDP is mutually exclusive with TCP and that TLS is mutually 
exclusive with DTLS. Couldn't that data be represented in two bits (one 
in which 0 ⇒ UDP and 1 ⇒ TCP, the other in which 0 ⇒ plain-text and 1 ⇒ 

On 02/22/2018 07:31 AM, Sara Dickinson wrote:
> Hi Folks,
> We have an update to draft which we hope captures all the comments to-date.
> - Make all data items in Q/R, QuerySignature and Malformed Message arrays optional
> - Re-structure the FilePreamble and ConfigurationParameters into BlockParameters
> - BlockParameters has separate Storage and Collection Parameters
> - Storage Parameters includes information on what optional fields are present, and flags specifying anonymisation or sampling
> - Addresses can now be stored as prefixes.
> - Switch to using a variable sub-second timing granularity
> - Add response bailiwick and query response type
> - Add specifics of how to record malformed messages
> - Add implementation guidance
> - Improve terminology and naming consistency
> There are still a number of ‘QUESTIONS’ in the draft that we would appreciate feedback on.
> Regards
> Sara.
>> On 22 Feb 2018, at 12:27, wrote:
>> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>> This draft is a work item of the Domain Name System Operations WG of the IETF.
>>         Title           : C-DNS: A DNS Packet Capture Format
>>         Authors         : John Dickinson
>>                           Jim Hague
>>                           Sara Dickinson
>>                           Terry Manderson
>>                           John Bond
>> 	Filename        : draft-ietf-dnsop-dns-capture-format-05.txt
>> 	Pages           : 62
>> 	Date            : 2018-02-22
>> Abstract:
>>    This document describes a data representation for collections of DNS
>>    messages.  The format is designed for efficient storage and
>>    transmission of large packet captures of DNS traffic; it attempts to
>>    minimize the size of such packet capture files but retain the full
>>    DNS message contents along with the most useful transport metadata.
>>    It is intended to assist with the development of DNS traffic
>>    monitoring applications.
>> The IETF datatracker status page for this draft is:
>> There are also htmlized versions available at:
>> A diff from the previous version is available at:
>> Please note that it may take a couple of minutes from the time of submission
>> until the htmlized version and diff are available at
>> Internet-Drafts are also available by anonymous FTP at:
>> _______________________________________________
>> DNSOP mailing list
> _______________________________________________
> DNSOP mailing list