Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt

Richard Gibson <rgibson@dyn.com> Wed, 05 July 2017 17:06 UTC

Return-Path: <rgibson@dyn.com>
X-Original-To: dnsop@ietfa.amsl.com
Delivered-To: dnsop@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84458129B3A for <dnsop@ietfa.amsl.com>; Wed, 5 Jul 2017 10:06:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=dyn.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W_1oJrx_Kp9G for <dnsop@ietfa.amsl.com>; Wed, 5 Jul 2017 10:06:34 -0700 (PDT)
Received: from mail-ua0-x248.google.com (mail-ua0-x248.google.com [IPv6:2607:f8b0:400c:c08::248]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E8383128B8F for <dnsop@ietf.org>; Wed, 5 Jul 2017 10:06:33 -0700 (PDT)
Received: by mail-ua0-x248.google.com with SMTP id y47so95233881uag.13 for <dnsop@ietf.org>; Wed, 05 Jul 2017 10:06:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dyn.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ZXf4XTZnv31/qc+Sz9ykMoD1cS/VUrVEDbVkir3YOWE=; b=W9jr6og8q4oMtsMqwtY+5a/Mx4pS7gIFrBBbzFbsqYTkwAfCaIqn0xC39wg7IyitiY AMRrjZ4pUCrUs9FTDPtozW9oLvFY6Xp6iOnqOIp1UPwbxhaZCqblAwy0rLEjDglOetQl bTQNhlSa0n0haBnCrXy0WOb7i1GGx5W4xivTk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ZXf4XTZnv31/qc+Sz9ykMoD1cS/VUrVEDbVkir3YOWE=; b=VbdzDUxaKidDfwM5VZigfFjvBEpYR4NYdN2Mjo+4t7eoLMccjsGoHg+c9kuysr5dyg VKkry7VBP77u5RKZlN8p8cuet7zMgFIgHQdYStwprsE3x/FAoz0lubjAZj55GS8iN3Bb Wf44ko5QL4csN9Ou2LZv3cvrU5y8LOmTkpAGCkdGcJbIWhX3o27sAtU9GFBsYyAUJZX6 0wiCZ55nZlJA1F5eT7onbpRM8amk36L2ZkgaVr11LrRlN/Q0JGeSea7evoVvmwV3mfFs 3imv5+obZcX56R1BgPK33Wd3VFxaFYczzGzO4S4JEMQKT4bWM+/gGw9qsHh4qmPP3Cqh IZJA==
X-Gm-Message-State: AKS2vOyevbRRQ/KnrjbK4eX7YOJr5lwul7LrJBrKRwdmnzwCP8THbVdY o6bVQ6feRqcGHob8EGgZY2ZHi3bHg2gEoUYRqg==
X-Received: by 10.31.124.66 with SMTP id x63mr25778884vkc.126.1499274393086; Wed, 05 Jul 2017 10:06:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.159.62.209 with HTTP; Wed, 5 Jul 2017 10:06:12 -0700 (PDT)
In-Reply-To: <5ec26bfa-b7c9-cdcc-2594-5e2df7bec4c8@sinodun.com>
References: <149907291397.4998.8059630450980375262@ietfa.amsl.com> <CAC94RYaY81Taq-iubcE+HRGGY7mLUAoLqSqFgyLWga5wCxfLSA@mail.gmail.com> <5ec26bfa-b7c9-cdcc-2594-5e2df7bec4c8@sinodun.com>
From: Richard Gibson <rgibson@dyn.com>
Date: Wed, 5 Jul 2017 13:06:12 -0400
Message-ID: <CAC94RYYBe5AzoBWEOeBrrBgKEcqw5V-z34TGtmcYpQW-JAz=KQ@mail.gmail.com>
To: Jim Hague <jim@sinodun.com>
Cc: dnsop <dnsop@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c14c0ec9f06cf055395053f"
Archived-At: <https://mailarchive.ietf.org/arch/msg/dnsop/Gj1cgPkYtdjMkZqjrmX5SqhbYw4>
Subject: Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt
X-BeenThere: dnsop@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IETF DNSOP WG mailing list <dnsop.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dnsop>, <mailto:dnsop-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dnsop/>
List-Post: <mailto:dnsop@ietf.org>
List-Help: <mailto:dnsop-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dnsop>, <mailto:dnsop-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Jul 2017 17:06:36 -0000

On Wed, Jul 5, 2017 at 8:05 AM, Jim Hague <jim@sinodun.com>; wrote:

> Timestamps, on the other hand, I always regarded as a basic data type,
> so naturally a structure. Plus, of course, there's one per
> query/response item, so in a block the size savings are in the 10-15k
> bytes region, which is rather more significant.
>

In that case, I'm even more convinced that all of them (block preamble
"earliest-time", Q/R data item "time" and "delay", malformed packet item
"time") should have the same structure—either variable-precision arrays or
{"seconds", "useconds", "pseconds"} maps.

> * In section 7.18, "and an unsigned key" appears to be meaningless and
> > should probably be removed.
>
> In most places where we are discussing a map, we've specified the type
> of the map key in the text, though I notice we're not 100% consistent
> with that.
>

All the keys in those maps (and in every map, as far as I can tell) are
strings, for which "unsigned" is a meaningless concept.


> Example use
> > cases:
> [...]
> > * Extend the block preamble (section 7.7) to override file preamble
> > fields like "host-id" and "server-addresses", enabling fleet-wide file
> > merges.
>
> I don't quite follow why you'd need to put this informational-only stuff
> into the block preamble rather than the file preamble/configuration. Can
> you expand on that a bit?
>

Some of our processing can benefit from merging telemetry (e.g., all hosts
at a site), which would produce files containing blocks that don't share
values for fields currently in the file preamble. In fact, we might even
decide to treat "block" as the only relevant unit, using block preamble
extension fields for *everything* in the standard file preamble except
version information.

> *Opt-in Lossyness*
> This raises a question about a tension between the background of C-DNS
> to date and the slightly different angle you are coming from. We've been
> very much focused on using C-DNS to record traffic in a form where the
> packets can be recreated in wire format (i.e. as PCAP). The optional
> data items mean that data may be missing from those packets, but the
> core query and response will still be present.
>

Agreed. But it's _so close_ that I felt these suggestions were worth
raising—not to water down the primary use case, but to cover a few more
with some tradeoffs.

moves the recording to a point where reconstructing wire format means
> that the application doing the reconstruction has to not just omit
> information not present in C-DNS, but must start generating values to
> fill in for the missing items. This feels a bit like a step that needs
> discussion; we need to think over the design from your point of view.
> Possibly those fields should be optional, but with recommendations for
> how to populate them when/if generating PCAP.
>

That sounds great.

We'll be doing a 10 minute slot on C-DNS in Prague, and would welcome
> discussion there.


I don't expect to be there personally, but we'll have a well-informed
representative.