Re: [apps-discuss] Describing JSON document formats

[Nico Williams <[e-mail]>]

On Fri, Nov 30, 2012 at 1:46 PM, Phillip Hallam-Baker <[e-mail]> wrote:
> On Fri, Nov 30, 2012 at 1:45 PM, Nico Williams <[e-mail]>
> wrote:
>> Well, type constraints are a good thing.  If you know that some number
>> will be 0..2^32 - 1 then you can use a uint32_t in a C implementation
>> -- at least you'll want to distinguish between integer sizes, bignums,
>> and so on.  Yeah, you don't need min/max to convey integer size.
>
> Representation constraints are useful, agreed.
>
> But these can be 8, 16, 32, 64 or BIgNum, signed or unsigned.

Sure.  Note that this can be done in ASN.1:

UInt32 ::= Integer (0..4294967295)

but that sort of thing kinda needs to be standard in order to function
properly as compiler hints.

> Agree on the languages and willing to contribute a compiler that can
> generate code for any of them.

I'd be willing to contribute to an open source compiler.  Since I like
ASN.1 and have contributed (minor stuff) to the Heimdal ASN.1
compiler, one idea that comes to mind is to have a JASN->ASN.1
transformation and then use ASN.1 compilers where one has them.

> Just got to get my ass into gear and work out how to use GIT in visual
> studio.

http://stackoverflow.com/questions/507343/using-git-with-visual-studio

>> But, really, something ASN.1-ish, minus the awful syntax, minus the
>> tags, and all that.  We might call it JASN, say (heh).
>
> Why limit it to JSON, can generate a subset of XML and ASN.1 as well.

Well, I didn't want to go there (given the animosity to ASN.1 'round
these parts), but IMO there's been nothing really new under the sun in
data encoding since S-expressions.  There have been new things
regarding data handling (XPath and XSLT, though I also like to think
that these are [very, very awesome] generalizations of LISP features
like destructuring-bind).  But abstract syntax notations are all
roughly equivalent.  Existence proof: XER and FastInfoSet.  (Even XDR
is really a subset of ASN.1 paired with a form of PER that is 4-octet
aligned and with some minor differences in the encoding of
absent/defaulted fields.)

So, yes: this JASN thing should be able to generate proper ASN.1.

One of the nice things about using JSON for the syntax is that it
should be naturally extensible.  If we start with a syntax geared for
just JSON today, then later want to be able to generate arbitrary
ASN.1, we'll need to be able to specify tags and such, and... that's
just new fields that old JASN parsers can just ignore.

> XML Scema was sufficiently bjorked that when I was writing the SAML spec I
> had to use a tool to generate the XML schema from something sensible.

I know, but really, XML Schema can be mapped onto what you want
anyways.  It's just more useful to have a syntax that more simply maps
onto programming language types to represent the same data.


> > [...]
>
> Ah now here we get to a pet peeve of mine. Extensibility. I don't think any
> of the protocols I have worked on has really had a good model.

No?  Kerberos has been extended extensively over the years -- it's not
been perfect, but it has worked out.

> The extensibility mechanisms in XML work fine for documents. For protocols
> they are a #$()WQFE&QWE #$$# !!!! pain in the ass. They just don't work and
> xxx:namespace yyy:prefixes are stupid.

Agreed.  For protocols we almost never need namespaces the way we do
for documents.

> [Validation stuff elided.]
>
> My problem is that none of the schema validation attributes has been
> sufficient for input validation. If I have a data item representing a cheque
> then the important thing is that the payment is less than the balance on
> that account, not that it is less than some static value in a schema.

That's semantic validation.  Yeah, it can't be done with an abstract
syntax like ASN.1 or XML Schema.  (One might here mention ITU-T's SDL,
but that's never been used in the IETF, there's no toolkits, and SDL
is borked anyways by being case-insensitive, which renders it
incompatible with ASN.1.)

> I agree that data input validation is important, just don't see that static
> schema constraints help.

Well, for config files it's a lot better than nothing, but yes, it
doesn't go far enough.

Nico
--