Re: [DNSOP] New draft on representing DNS messages in JSON

[Jay Daley <[e-mail]>]

I agree with all of Shane's points below, namely better encoding of data, not making all fields optional, handling of unknown rrtypes and having a schema for the data.  These are all points (and many more) that were thoroughly addressed in the following draft on representing DNS in XML

	http://tools.ietf.org/html/draft-daley-dnsxml-00

If it helps then I can produce XSLT that takes data in the XML format specified in that draft and validated against that schema and turn it into whatever flavour of JSON you require:

	http://controlfreak.net/xml-to-json-in-xslt-a-toolkit/

*stands back and waits for the "but json is sooo much easier than xml" brickbats*

Jay


On 22/08/2014, at 3:27 am, Shane Kerr <[e-mail]> wrote:

> Paul,
> 
> On Thu, 21 Aug 2014 07:01:01 -0700
> Paul Hoffman <[e-mail]> wrote:
> 
>> On Aug 21, 2014, at 1:22 AM, Shane Kerr <[e-mail]>
>> wrote:
>> 
>>> * I don't like the treatment of QNAME*/hostQNAME, NAME*/hostNAME,
>>> and so on. Since JSON includes encoded strings, wouldn't it make
>>> more sense just to always put the QNAME in there? (Especially since
>>> you'll end up with SRV queries always being encoded as they have
>>> underscore characters...)
>> 
>> JSON requires its strings to be encoded in a particular character
>> set. Given that the labels in a QNAME/NAME can be an binary cruft,
>> you can't assume that every QNAME will be representable.
> 
> I think you're making it too hard. Control characters, ", and \ are
> already required to be escaped. Just specify a similar requirement for
> octets 127 to 255 also be escaped, and we're done.
> 
>>> * In general I'm not super enthusiastic about the mixing of binary
>>> and formatted data - I tend to think an application will want one
>>> or the other. Perhaps it makes more sense to define two formats,
>>> one binary and one formatted? Or...
>> 
>> All fields are optional, so a profile could say "don't include these"
>> or "always include those". Further, and more importantly, most RDATA
>> are binary. I did not want to force implementations to use the
>> presentation format for RDATA.
> 
> The problem with an "all fields are optional" approach is that it puts
> all the burden on the consumer of the data, right? You literally have
> no idea what to expect. (That's kind of why I proposed some sort of
> schema below.)
> 
> I understand not wanting to force implementations to use the
> presentation format for RDATA... OTOH it seems likely that the reason
> people are putting data in JSON is so they can see what it is. We could
> always try the RFC 3597 approach for an unknown RTYPE?
> 
>>> * Maybe it makes sense to define a meta-record so consumers can know
>>> what to expect? Something that lists which names will (or may)
>>> appear.
>> 
>> That would be a JSON schema. Just using that phrase will cause
>> screaming in the Apps Area. Having said that, it's perfectly
>> reasonable for a profile to insist that each record have a profile
>> indicator such as "Profile": "Private DNS interchange v3.1".
> 
> Screaming aside, applications will either have an implicit schema or an
> explicit one. Defining the problem to be out of scope may be necessary
> to get something published, but that's a symptom of IETF brokenness
> IMHO, since it reduces the usefulness of any such RFC. :(
> 
>>> I'd be mildly curious to see a comparison of the compressed sizes of
>>> JSON-formatted data (without data duplicated as binary stuff) versus
>>> non-JSON-formatted data. My intuition is that compression will
>>> remove most of the horrible redundancy that is involved in JSON,
>>> but there's only one way to be sure. ;)
>> 
>> Sure. It's pretty trivial to do, for example, a CBOR format that
>> follows this; there are now CBOR libraries for most popular modern
>> languages (see http://cbor.io/) If folks here want that, I can add
>> it as an appendix. To be clear, however, I haven't heard anyone
>> saying they want compression so badly they are willing to lose
>> readability of the data.
> 
> Oh, I meant with gzip or the like, not some JSON crafted format.
> 
> So the idea is:
> 
>   $ tcpdump -w somefile.pcap
>   $ pcap2dnsjson somefile.pcap somefile.json
>   $ gzip somefile.pcap
>   $ gzip somefile.json
>   $ ls -l somefile.{pcap,json}.gz
> 
> Then compare the sizes of the compressed files.
> 
> The idea being that when moving files around via scp or rsync or
> whatever they'd probably be compressed like this, and probably also for
> medium-term storage. My hope is that a compressed JSON is roughly the
> same size as a compress raw pcap file, since basically they have the
> same entropy.
> 
> The reason I bring this up is to give a feel for the size cost of a
> bloated text format in practice. :)
> 
> Cheers,
> 
> --
> Shane
> 
> _______________________________________________
> DNSOP mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dnsop


-- 
Jay Daley
Chief Executive
.nz Registry Services (New Zealand Domain Name Registry Limited)
desk: [phone]
mobile: [phone]
linkedin: www.linkedin.com/in/jaydaley