Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-05.txt

[Richard Gibson <[e-mail]>]

Thank you so much for the changes, I am confident that this format is 
now in a state where my organization could build on top of it to suit 
our needs. I have a handful of remaining comments below, mostly 
describing ways in which that process could be made more straightforward 
and standardized.

* Section 6.2.1 and 7.5.3.4: At minimum, RR field "ttl" should be 
optional in order to accommodate aggregation of resolver responses in 
which that field is decremented over time. But probably "rdata-index" 
should be optional as well.

* Section 7.1: Negative integer map keys are concise, but they come with 
the potential for interoperability-hostile collisions. I think the 
document should encourage restricting their use to tightly-controlled 
closed systems, and then additionally propose a particular namespacing 
pattern (e.g., "{namespaceName}-{keyName}") for use of 
implementation-specific /string/ keys in other cases. Further, it should 
explicitly reserve positive integer keys for future standardized 
extensions to the C-DNS format.

* Section 7.4.1: Why is BlockParameters an array instead of a map?

* Section 7.4.1.1: StorageParameters fields "opcodes" and "rrtypes" 
could be defined better... are they values /recorded/ (i.e., that were 
actually observed) or values that are /recordable/ (i.e., that the 
collection application was looking for)? And shouldn't they be optional 
(in either case, but *especially* the latter, lest implementations write 
out every possible rrtype value).

* Section 7.5.3: It's very good that the BlockTables "ip-address" items 
can be truncated by prefix length, but we have actually run into cases 
that call for /variable/ truncation (i.e., more precision in some 
subnets than in others), especially true given that a single table holds 
both client and server address data. What would you think about 
representing addresses not as direct byte strings, but rather as 
variable-length arrays or maps—index 0 encoding the prefix as a byte 
string, and optional index 1 (when present) encoding a prefix-length 
override? Or, if the extra size per address is too much, at least 
allowing such representations via polymorphism (byte string for default 
prefix-length, array/map for non-default)?

* Section 7.5.3.2: QueryResponseSignature field "qr-transport-flags" 
seems to reserve 1 bit /each/ for UDP, TCP, TLS, and DTLS, but it seems 
to me that UDP is mutually exclusive with TCP and that TLS is mutually 
exclusive with DTLS. Couldn't that data be represented in two bits (one 
in which 0 ⇒ UDP and 1 ⇒ TCP, the other in which 0 ⇒ plain-text and 1 ⇒ 
TLS)?


On 02/22/2018 07:31 AM, Sara Dickinson wrote:
> Hi Folks,
>
> We have an update to draft which we hope captures all the comments to-date.
>
> - Make all data items in Q/R, QuerySignature and Malformed Message arrays optional
> - Re-structure the FilePreamble and ConfigurationParameters into BlockParameters
> - BlockParameters has separate Storage and Collection Parameters
> - Storage Parameters includes information on what optional fields are present, and flags specifying anonymisation or sampling
> - Addresses can now be stored as prefixes.
> - Switch to using a variable sub-second timing granularity
> - Add response bailiwick and query response type
> - Add specifics of how to record malformed messages
> - Add implementation guidance
> - Improve terminology and naming consistency
>
> There are still a number of ‘QUESTIONS’ in the draft that we would appreciate feedback on.
>
> Regards
>
> Sara.
>
>> On 22 Feb 2018, at 12:27, [e-mail] wrote:
>>
>>
>> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>> This draft is a work item of the Domain Name System Operations WG of the IETF.
>>
>>         Title           : C-DNS: A DNS Packet Capture Format
>>         Authors         : John Dickinson
>>                           Jim Hague
>>                           Sara Dickinson
>>                           Terry Manderson
>>                           John Bond
>> 	Filename        : draft-ietf-dnsop-dns-capture-format-05.txt
>> 	Pages           : 62
>> 	Date            : 2018-02-22
>>
>> Abstract:
>>    This document describes a data representation for collections of DNS
>>    messages.  The format is designed for efficient storage and
>>    transmission of large packet captures of DNS traffic; it attempts to
>>    minimize the size of such packet capture files but retain the full
>>    DNS message contents along with the most useful transport metadata.
>>    It is intended to assist with the development of DNS traffic
>>    monitoring applications.
>>
>>
>> The IETF datatracker status page for this draft is:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dietf-2Ddnsop-2Ddns-2Dcapture-2Dformat_&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=522TfZMUJHLQox2FWFORwOQXhrlqwp36X9Ty1v54YeI&e=
>>
>> There are also htmlized versions available at:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Ddnsop-2Ddns-2Dcapture-2Dformat-2D05&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=Hc60T3mAF7VZgnUYR0LyhsTUTloEhCJIH3wIfUyCYmU&e=
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_html_draft-2Dietf-2Ddnsop-2Ddns-2Dcapture-2Dformat-2D05&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=oT9ecpv59bEHyd7Tpu69VbCmXsTBJYi3ePPp8kkuQGc&e=
>>
>> A diff from the previous version is available at:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_rfcdiff-3Furl2-3Ddraft-2Dietf-2Ddnsop-2Ddns-2Dcapture-2Dformat-2D05&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=WAyfWMq1ADNBRPEAlbUe-vsBwe1E7XGilSC9g498WkM&e=
>>
>>
>> Please note that it may take a couple of minutes from the time of submission
>> until the htmlized version and diff are available at tools.ietf.org.
>>
>> Internet-Drafts are also available by anonymous FTP at:
>> https://urldefense.proofpoint.com/v2/url?u=ftp-3A__ftp.ietf.org_internet-2Ddrafts_&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=RS6uAkaKhYyD4CDe2_bX21qAvs0hkRPcFZ02B5wk_Kc&e=
>>
>> _______________________________________________
>> DNSOP mailing list
>> [e-mail]
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_dnsop&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=8dC7W8NvScBDakKiehjmI17kYXh39QFcg488yknExDo&e=
> _______________________________________________
> DNSOP mailing list
> [e-mail]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_dnsop&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=-o8MJF7i0TzXAJRB0ncfTVfWKSyTG7nl_iTLU_A2B7c&m=S0pguE4aKgWT-7tOAVBInPr7s58myqhSmv9WtwMzeNA&s=8dC7W8NvScBDakKiehjmI17kYXh39QFcg488yknExDo&e=