Re: [Cbor] CDDL parsing questions

[Carsten Bormann <[e-mail]>]

Hi Toerless,

You are trying to on-the-fly create a processing model for CDDL.
This requires a bit more thinking and critical design.

>> I prefer to reserve the term “parsing” to text-based protocols that are best handled with parser generators.
> 
> Except for "text-based protocols", that was exactly what i was thinking of - if
> i correctly understand you:
> 
> "CDDL parser:"

A program that parses CDDL.  Not something that happens on a constrained device at run time.  Example: [1]

[1]: https://www.ietf.org/archive/id/draft-bormann-cbor-cddl-freezer-09.html#name-alternative-representations

> A program that creates from CDDL input a program which takes a CBOR input
> and spits out a structure (tree?) of CDDL names, each pointing to the "parsed"
> CDDL structures that it represents.

That would be a “validator generator”, with some concept of PSVI (post-schema-validation instance) that includes annotation by rule names.
A very specific processing model that may not at all fit all sizes.

> CVE ?

https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures

>> What you seem to be alluding to, is the ingestion of a CBOR data item in CBOR generic data model into the semantic categories that application wants.  CDDL can describe some, but not all of this process.
> 
> My main point is not even about implementation, but see above.
> 
> My main point is that if we use CDDL to specify protocol structures with CDDL
> names that we then need an agreement about what it means for protocol input/output
> to comply with that CDDL specification or not.

That depends on what you said in the specification.
Typically, the specification would require certain messages to match certain CDDL rules.
(Either by making the CDDL normative or by saying there is a normative English language specification that is then in practice made comprehensible and disambiguated by the CDDL, just like we have been using ABNF in many cases.)

> To me, that is the case if i
> could have the above "CDDL parser" and it would take my CBOR protocol structure input
> and attach the CDDL name to it that i think that CBOR protocol structure represents.

This presumes a processing model that may not be common to all implementations of a protocol; i.e., this would be overspecification.

> Not really any different whether i specify in CDDL or in ASCII-art, only
> that i think we never philosophized about the process of determining whether or
> not a protocol structure is compliant with the specification - because we
> intuitively/from-experience always choose to define protocol structures
> simple enough that we didn't have much to discuss.

We didn’t have to, because “Protocol message matches CDDL rule” is a very straightforward rule.

>> CDDL can be used to write complex grammars that require more look-ahead than one would like to have, e.g.
>> 
>> Message = Message1 / Message2
>> 
>> Message1 = [foo, bar, 1]
>> 
>> Message2 = [foo, bar, 2]
>> 
>> Don’t do that.
> 
> Exactly. This is what i think our "CDDL protocol" in question does, or
> at least would do if we went down that path, and hence this mailing list thread.

But this is a quality-of-specification discussion, not a what-does-the-specification-mean question.

>> (A tool that flags excessive look-ahead requirements would be useful.
>> In this case, putting the discriminator up front is helpful:
>> Message1 = [1, foo, bar]
>> Message2 = [2, foo, bar]
> 
> Exactly. But IMHO that is ONLY necessary/benefical if we do have a good
> definition as to what "CDDL protocols" can and which ones can't afford this lookahead
> 
>  good-protocol = [This, is, a, lovely, protocol, ",", dear]
>  bad-protocol  = [This, is, a, lovely, protocol, ",", idiot]

We don’t have to (and, really, can’t) define a quality threshold.
Having a spec that also explains quality aspects such as avoiding unnecessary look-ahead would be useful.
This could be done in conjunction with a more details examination of processing models that I would expect to be part of the CDDL 2.0 effort.

> For on-the-wire-protocols i don't think i have ever seen this "lookahead",

I’m pretty sure I’ve had to look into parameters inside TLVs to categorize the things that had those parameters; this is a form of look-ahead.

> but in programming and human language parsers it is of course common.
> 
> So now i fundamentally start to wonder if we're not missing out on wonderful
> world of richer, and for some reason better syntax in on-the-wire protocols
> solely because previously we designed on-the-wire protocols primarily
> so that "hand-written-parsers" could be easy, whereas those human/computer-language
> parsers already went way beyond that layer of the problem and had "automated"
> the parsing, hence achieving far more flexible syntax.

CBOR was designed to enable a very, very automatic lexical level (generic decoder), and to allow the whole gamut of approaches to the syntactic (structural) level, including hand-coding, code generation, and generation of parameters for a more generic (“table-driven”) syntactic decoder.  Also, the two levels can be munched up if that helps (it usually does with memory allocation).

> But then of course i go back and ask: what is the most simple _good_
> example why we would want to do lookahead. Right now our protocol in question
> answer to me is a bit "we forgot to avoid lookahead in our original design,
> and when we now want to extend the protocol with maximum backward
> compatibility, we create lookahead". But i am nor persuaded that this is
> agood-enough reason.

I don’t think this can be answered in a general way.

Grüße, Carsten