Re: [DNSOP] I-D Action: draft-ietf-dnsop-dns-capture-format-03.txt

[Jim Hague <[e-mail]>]

On Wednesday, 5 July 2017 13:06:12 BST Richard Gibson wrote:
> On Wed, Jul 5, 2017 at 8:05 AM, Jim Hague <[e-mail]> wrote:
> > Timestamps, on the other hand, I always regarded as a basic data type,
> > so naturally a structure. Plus, of course, there's one per
> > query/response item, so in a block the size savings are in the 10-15k
> > bytes region, which is rather more significant.
> 
> In that case, I'm even more convinced that all of them (block preamble
> "earliest-time", Q/R data item "time" and "delay", malformed packet item
> "time") should have the same structure—either variable-precision arrays or
> {"seconds", "useconds", "pseconds"} maps.

Q/R data item and malformed packet items have the same structure, though it's 
not an explicit data type in CDDL as Timeval is at the moment. I'll amend 
that. However, they are time deltas, not timestamps, so I don't think they 
should have the same type as the block earliest time. It it turns out that no-
better-then-second resolution is a common use case, there are potential 
savings to be made in using seconds as the time delta, because the deltas will 
tend to be very small numbers with consequent space savings in CDDL. We'll 
look at this.

> > * In section 7.18, "and an unsigned key" appears to be meaningless and 
> > > should probably be removed.
> > 
> > In most places where we are discussing a map, we've specified the type
> > of the map key in the text, though I notice we're not 100% consistent
> > with that.
> 
> All the keys in those maps (and in every map, as far as I can tell) are
> strings, for which "unsigned" is a meaningless concept.

No. All keys are unsigned ints, with values specified in the CDDL. We should 
make this more explicit in the text.

String keys would bloat the output file size enormously.

> > I don't quite follow why you'd need to put this informational-only stuff
> > into the block preamble rather than the file preamble/configuration. Can
> > you expand on that a bit?
> 
> Some of our processing can benefit from merging telemetry (e.g., all hosts
> at a site), which would produce files containing blocks that don't share
> values for fields currently in the file preamble. In fact, we might even
> decide to treat "block" as the only relevant unit, using block preamble
> extension fields for *everything* in the standard file preamble except
> version information.

Ah, so you are talking about doing a merge at the block level rather than the 
Q/R item level? We'll think about the implications of that.

The fields you cited, host-id and server-addresses, are purely informational 
anyway.

> > *Opt-in Lossyness*
> > This raises a question about a tension between the background of C-DNS
> > to date and the slightly different angle you are coming from. We've been
> > very much focused on using C-DNS to record traffic in a form where the
> > packets can be recreated in wire format (i.e. as PCAP). The optional
> > data items mean that data may be missing from those packets, but the
> > core query and response will still be present.
> 
> Agreed. But it's _so close_ that I felt these suggestions were worth
> raising—not to water down the primary use case, but to cover a few more
> with some tradeoffs.

Oh, absolutely! You've give an important use case we hadn't considered - we 
need to look into it.

> We'll be doing a 10 minute slot on C-DNS in Prague, and would welcome
> > discussion there.
> 
> I don't expect to be there personally, but we'll have a well-informed
> representative.

We'd definitely like to talk, then.
-- 
Jim Hague - [e-mail]          Never trust a computer you can't lift.