Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\))
From: Paul Hoffman <paul.hoffman@vpnc.org>
In-Reply-To: <20140821172757.504f49b7@vulcan>
Date: Thu, 21 Aug 2014 09:11:18 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <97A31C1B-329D-44A3-A7C5-888F9E10C38D@vpnc.org>
References: <B4ACD73A-25EF-4063-81D4-DCFE6DB78AB1@vpnc.org>
 <20140821102232.73071610@vulcan>
 <586AEB36-C10F-4E6E-AC55-BAADE7C00FD4@vpnc.org>
 <20140821172757.504f49b7@vulcan>
To: Shane Kerr <shane@time-travellers.org>
Archived-At: http://mailarchive.ietf.org/arch/msg/dnsop/4eoXijWdUex_eEOyvDwWrYK4vic
Cc: dnsop <dnsop@ietf.org>
Subject: Re: [DNSOP] New draft on representing DNS messages in JSON
Precedence: list

On Aug 21, 2014, at 8:27 AM, Shane Kerr <shane@time-travellers.org> =
wrote:

>>> * In general I'm not super enthusiastic about the mixing of binary
>>> and formatted data - I tend to think an application will want one
>>> or the other. Perhaps it makes more sense to define two formats,
>>> one binary and one formatted? Or...
>>=20
>> All fields are optional, so a profile could say "don't include these"
>> or "always include those". Further, and more importantly, most RDATA
>> are binary. I did not want to force implementations to use the
>> presentation format for RDATA.
>=20
> The problem with an "all fields are optional" approach is that it puts
> all the burden on the consumer of the data, right? You literally have
> no idea what to expect. (That's kind of why I proposed some sort of
> schema below.)

Correct.

> I understand not wanting to force implementations to use the
> presentation format for RDATA... OTOH it seems likely that the reason
> people are putting data in JSON is so they can see what it is. We =
could
> always try the RFC 3597 approach for an unknown RTYPE?

Define "unknown". Note that at least one "known" RTYPE has its =
description wrong, but you'd only know that if you read the RFC errata.

I could have RDATA is a string that is in presentation format, and hope =
that all emitters and receivers get that right, with an optional =
additional filed of rdataOctets! for a base64url-encoded version. If the =
emitter doesn't know the presentation form, it uses that only; if the =
emitter isn't sure that the receiver knows the presentation form, the =
emitter adds an rdataOctets! field as well. If the emitter knows that =
the RDATA is broken (such as a three-octet value for an A record), it =
just uses an rdataOctets! field. Would that suffice for you?=20

>>> * Maybe it makes sense to define a meta-record so consumers can know
>>> what to expect? Something that lists which names will (or may)
>>> appear.
>>=20
>> That would be a JSON schema. Just using that phrase will cause
>> screaming in the Apps Area. Having said that, it's perfectly
>> reasonable for a profile to insist that each record have a profile
>> indicator such as "Profile": "Private DNS interchange v3.1".
>=20
> Screaming aside, applications will either have an implicit schema or =
an
> explicit one. Defining the problem to be out of scope may be necessary
> to get something published, but that's a symptom of IETF brokenness
> IMHO, since it reduces the usefulness of any such RFC. :(

The IETF looked at the horrid mess crated by XML schema, and decided =
that implicit schema described in text are better. And before you attack =
the messenger on this, note that (a) I tried to get the IETF to look at =
JSON schema again and (b) I'm the co-chair of the JSON WG.

>>> I'd be mildly curious to see a comparison of the compressed sizes of
>>> JSON-formatted data (without data duplicated as binary stuff) versus
>>> non-JSON-formatted data. My intuition is that compression will
>>> remove most of the horrible redundancy that is involved in JSON,
>>> but there's only one way to be sure. ;)
>>=20
>> Sure. It's pretty trivial to do, for example, a CBOR format that
>> follows this; there are now CBOR libraries for most popular modern
>> languages (see http://cbor.io/). If folks here want that, I can add
>> it as an appendix. To be clear, however, I haven't heard anyone
>> saying they want compression so badly they are willing to lose
>> readability of the data.
>=20
> Oh, I meant with gzip or the like, not some JSON crafted format.
>=20
> So the idea is:
>=20
>   $ tcpdump -w somefile.pcap
>   $ pcap2dnsjson somefile.pcap somefile.json
>   $ gzip somefile.pcap
>   $ gzip somefile.json
>   $ ls -l somefile.{pcap,json}.gz
>=20
> Then compare the sizes of the compressed files.
>=20
> The idea being that when moving files around via scp or rsync or
> whatever they'd probably be compressed like this, and probably also =
for
> medium-term storage. My hope is that a compressed JSON is roughly the
> same size as a compress raw pcap file, since basically they have the
> same entropy.
>=20
> The reason I bring this up is to give a feel for the size cost of a
> bloated text format in practice. :)

Noted. That's an easy thing for implementers of this format to do. So is =
saying that size is so important (maybe because we finally have so many =
DNSSEC keys passed about) that CBOR or something like it would be =
better.

--Paul Hoffman=