Return-Path: X-Original-To: dnsop@ietfa.amsl.com Delivered-To: dnsop@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 173471317BF for ; Mon, 3 Jul 2017 16:22:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=dyn.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Thg24hDErl6m for ; Mon, 3 Jul 2017 16:22:41 -0700 (PDT) Received: from mail-ua0-x248.google.com (mail-ua0-x248.google.com [IPv6:2607:f8b0:400c:c08::248]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E6A61317B2 for ; Mon, 3 Jul 2017 16:22:36 -0700 (PDT) Received: by mail-ua0-x248.google.com with SMTP id j53so77215789uaa.2 for ; Mon, 03 Jul 2017 16:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dyn.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0B2zkxjxvfobK/XfBhGR4Qcj4Jg7qscXF8mXglUKcx0=; b=H+IQq4ujTnWbnC6R88PASySf3F9oLPB/G2VGmfPnuM96DbSAZvi+QlXm3nlGMqeQn+ j/a4vD1N5CTA8ty3irwpDTmvm0l5bbNRTR11ASHkbRqidOCX4GAo6P4kxepRwZn8hu0M KkaHTKnFwGO58ksF5Hh9boeD+qEA1gFSgW2DQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0B2zkxjxvfobK/XfBhGR4Qcj4Jg7qscXF8mXglUKcx0=; b=XIbV0RTJEwmPE/Q8AwAuvI+29a23vuN9hkFts3gO350PkbhvcHsa08pKKwXFJDVlO4 a7pcPAJJH0D/p0VcvGI+DhpQPOdtw7W+Dkne83XAUl/2BWH40MHnttyUUHv0RktOfSeB Ls4SAEhe1Pc8fibPx1+EujCA5ZSCNlhly9U5ObLqwCXCradSfAlmjsKN4LlwlGp09lEg EkmHNS9yPjuTspIk/XBr9+u96a0+rP5SoeDWiKcwbZTvCvAZtmC6V8y8Q+iKUcKbWrYu 5benZF1YKBcFOM+GqpJ4MX/A9OtoZ4am/2+vbD+vCwCwqW3pYZAmrOZQfuN6wh2B87HY wC+Q== X-Gm-Message-State: AKS2vOzxoXLEWEnqASFzcDdJLNQiXbe+FeruYW9LVsPu0Bk2/VqBS4V2 KJKL7h2BeMQU3XnG9wN2TgjgMeg9/AWP X-Received: by 10.159.59.18 with SMTP id i18mr20289617uah.62.1499124155163; Mon, 03 Jul 2017 16:22:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.62.209 with HTTP; Mon, 3 Jul 2017 16:22:14 -0700 (PDT) In-Reply-To: <149907291397.4998.8059630450980375262@ietfa.amsl.com> References: <149907291397.4998.8059630450980375262@ietfa.amsl.com> From: Richard Gibson Date: Mon, 3 Jul 2017 19:22:14 -0400 Message-ID: To: internet-drafts@ietf.org Cc: i-d-announce@ietf.org, dnsop Content-Type: multipart/alternative; boundary="f403043c449cbe3bf30553720a8f" Archived-At: Subject: Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt X-BeenThere: dnsop@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: IETF DNSOP WG mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jul 2017 23:22:50 -0000 --f403043c449cbe3bf30553720a8f Content-Type: text/plain; charset="UTF-8" I looked over this draft in detail, and found a handful of ambiguous points ("Clarifications" and "Potentially Missing Data" below). But more importantly, it is very close to defining a format that could replace much of my organization's in-house technology. Would you consider some generalizations to take it over the finish line ("Extension Fields" and "Opt-in Lossyness")? Only the suggestions related to representing time and "classtype" items would change the representation of existing data in such a way that implementations already supporting the draft specification would require changes. *Clarifications* * Items in the "classtype" table (section 7.11) are missing data type documentation. Both "type" and "class" should be unsigned numbers. * And speaking of 7.11, why are CLASS/TYPE pairs represented as CBOR maps instead of more efficient two-item arrays? If it was an intentional decision for clarity, then maybe the section 7.7 block preamble "earliest-time" field should also be promoted to a map ("time-seconds", "time-useconds", "time-pseconds", mirroring Q/R items) for the same reason. * In "query-sig" table items (section 7.13) "transport-flags" field, the bit corresponding to "trailing bytes" shouldn't be limited to UDP. * In section 7.18, "and an unsigned key" appears to be meaningless and should probably be removed. *Potentially Missing Data* * In "query-sig" table items (section 7.13), "transport-flags" should probably be extended to include a TLS bit (cf. RFC 7858). *Extension Fields* Of the many potentially open-ended key-value maps (file preamble, file preamble configuration, block preamble, block statistics, query signatures, Q/R data), only block statistics allows for "implementation-specific fields", and no further guidance is provided. I think all maps should allow such fields, with a recommendation that they use an implementation-specific prefix to avoid collisions with fields added by other implementations or later versions of C-DNS. Example use cases: * Extend file preamble configuration (section 7.5) to document aggregation of queries answered by wildcard names. * Extend the block preamble (section 7.7) to override file preamble fields like "host-id" and "server-addresses", enabling fleet-wide file merges. * Extend query signatures (section 7.13) to indicate EDNS Client Subnet use (cf. RFC 7871) or selection of context-dependent handling like views. * Extend Q/R data (section 7.18) to include actual EDNS Client Subnet values or even a "count" field (which would come with *tremendous* space savings, sufficient for my organization to abandon its internal summarization format in favor of C-DNS). *Opt-in Lossyness* The format is generally quite good about allowing for detail without requiring it. However, there are some areas where more space savings could be had: * Communicate aggregation of IP addresses into prefixes (i.e., the irrelevance of least-significant bits in ip-address values) with new "client-prefix-length-ipv4" and "client-prefix-length-ipv6" and "server-prefix-length-ipv4" and "server-prefix-length-ipv6" file preamble configuration options. * Communicate case-normalizing aggregation of names (e.g., transforming "eXaMpLe.com" into "example.com") with a new boolean-valued "name-normalization" file preamble configuration option. * In "rr" table items (section 7.15), "ttl" should be optional to accommodate decrementing in recursive resolver responses. * In Q/R data items (section 7.18) and malformed packet records (section 7.20), I'd like "time-useconds" broken out into "time-seconds" and optional "time-useconds", both for parity with block-preamble "earliest-time" and for space savings in applications that are content with second-level resolution. * For truly customizable aggregation, I think all query signature (section 7.13) and Q/R (section 7.18) data item fields should be optional... but especially Q/R data "client-port" and "transaction-id". On Mon, Jul 3, 2017 at 5:08 AM, wrote: > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > This draft is a work item of the Domain Name System Operations of the IETF. > > Title : C-DNS: A DNS Packet Capture Format > Authors : John Dickinson > Jim Hague > Sara Dickinson > Terry Manderson > John Bond > Filename : draft-ietf-dnsop-dns-capture-format-03.txt > Pages : 49 > Date : 2017-07-03 > > Abstract: > This document describes a data representation for collections of DNS > messages. The format is designed for efficient storage and > transmission of large packet captures of DNS traffic; it attempts to > minimize the size of such packet capture files but retain the full > DNS message contents along with the most useful transport metadata. > It is intended to assist with the development of DNS traffic > monitoring applications. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/ > > There are also htmlized versions available at: > https://tools.ietf.org/html/draft-ietf-dnsop-dns-capture-format-03 > https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-dns- > capture-format-03 > > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=draft-ietf-dnsop-dns-capture-format-03 > > > Please note that it may take a couple of minutes from the time of > submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > DNSOP mailing list > DNSOP@ietf.org > https://www.ietf.org/mailman/listinfo/dnsop > --f403043c449cbe3bf30553720a8f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I looked over this draft in detail, and found a handf= ul of ambiguous points ("Clarifications" and "Potentially Mi= ssing Data" below). But more importantly, it is very close to defining= a format that could replace much of my organization's in-house technol= ogy. Would you consider some generalizations to take it over the finish lin= e ("Extension Fields" and "Opt-in Lossyness")? Only the= suggestions related to representing time and "classtype" items w= ould change the representation of existing data in such a way that implemen= tations already supporting the draft specification would require changes.
Clarifications
* Items in the "classtype" table (section 7.11) a= re missing data type documentation. Both "type" and "class&q= uot; should be unsigned numbers.
=C2=A0 * And speaking of 7.11, w= hy are CLASS/TYPE pairs represented as CBOR maps instead of more efficient = two-item arrays? If it was an intentional decision for clarity, then maybe = the section 7.7 block preamble "earliest-time" field should also = be promoted to a map ("time-seconds", "time-useconds", = "time-pseconds", mirroring Q/R items)=C2=A0for the same reason.
* In "query-sig" table items (section 7.13) "transp= ort-flags" field, the bit corresponding to "trailing bytes" = shouldn't be limited to UDP.
* In section 7.18, "and= an unsigned key" appears to be meaningless and should probably be rem= oved.

Potentially Missing Data
* = In "query-sig" table items (section 7.13), "transport-flags&= quot; should probably be extended to include a TLS bit (cf. RFC=C2=A07858).=

Extension Fields
Of the many= potentially open-ended key-value maps (file preamble, file preamble config= uration, block preamble, block statistics, query signatures, Q/R data), onl= y block statistics allows for "implementation-specific fields", a= nd no further guidance is provided. I think all maps should allow such fiel= ds, with a recommendation that they use an implementation-specific prefix t= o avoid collisions with fields added by other implementations or later vers= ions of C-DNS. Example use cases:
* Extend file preamble configuration (= section 7.5) to document aggregation of queries answered by wildcard names.=
* Extend the block preamble (section 7.7) to override file pream= ble fields like "host-id" and "server-addresses", enabl= ing fleet-wide file merges.
* Extend query signatures (section 7.= 13) to indicate EDNS Client Subnet use (cf. RFC 7871) or selection of conte= xt-dependent handling like views.
* Extend Q/R data (section 7.18= ) to include actual EDNS Client Subnet values or even a "count" f= ield (which would come with tremendous=C2=A0space savings, sufficien= t for my organization to abandon its internal summarization format in favor= of C-DNS).

Opt-in Lossyness
The = format is generally quite good about allowing for detail without requiring = it. However, there are some areas where more space savings could be had:
* Communicate aggregation of IP addresses into prefixes (i.e., the = irrelevance of least-significant bits in ip-address values) with new "= client-prefix-length-ipv4" and "client-prefix-length-ipv6" a= nd "server-prefix-length-ipv4" and "server-prefix-length-ipv= 6" file preamble configuration options.
* Communicate case-n= ormalizing aggregation of names (e.g., transforming "eXaMpLe.com"= into "example.com") with a ne= w boolean-valued "name-normalization" file preamble configuration= option.
* In "rr" table items (section 7.15), "tt= l" should be optional to accommodate decrementing in recursive resolve= r responses.
* In Q/R data items (section 7.18) and malformed pac= ket records (section 7.20), I'd like "time-useconds" broken o= ut into "time-seconds" and optional "time-useconds", bo= th for parity with block-preamble "earliest-time" and for space s= avings in applications that are content with second-level resolution.
=
* For truly customizable aggregation, I think all query signature (sec= tion 7.13) and Q/R=C2=A0(section 7.18)=C2=A0data item fields should be opti= onal... but especially Q/R data "client-port" and "transacti= on-id".

On Mon, Jul 3, 2017 at 5:08 AM, <internet-drafts@ietf.org> wrote:

A New Internet-Draft is available from the on-line Internet-Drafts director= ies.
This draft is a work item of the Domain Name System Operations of the IETF.=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Title=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0:= C-DNS: A DNS Packet Capture Format
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Authors=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: John= Dickinson
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Jim Hague
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Sara Dickinson
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Terry Manderson
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 John Bond
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Filename=C2=A0 =C2=A0 =C2=A0 =C2=A0 : draft-iet= f-dnsop-dns-capture-format-03.txt
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Pages=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0:= 49
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Date=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 := 2017-07-03

Abstract:
=C2=A0 =C2=A0This document describes a data representation for collections = of DNS
=C2=A0 =C2=A0messages.=C2=A0 The format is designed for efficient storage a= nd
=C2=A0 =C2=A0transmission of large packet captures of DNS traffic; it attem= pts to
=C2=A0 =C2=A0minimize the size of such packet capture files but retain the = full
=C2=A0 =C2=A0DNS message contents along with the most useful transport meta= data.
=C2=A0 =C2=A0It is intended to assist with the development of DNS traffic =C2=A0 =C2=A0monitoring applications.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/

There are also htmlized versions available at:
https://tools.ietf.org/html/d= raft-ietf-dnsop-dns-capture-format-03
https://datatracker.ietf= .org/doc/html/draft-ietf-dnsop-dns-capture-format-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdi= ff?url2=3Ddraft-ietf-dnsop-dns-capture-format-03


Please note that it may take a couple of minutes from the time of submissio= n
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

--f403043c449cbe3bf30553720a8f--