Re: [abnf-discuss] defining "compatible" extensions

[Paul Kyzivat <[e-mail]>]

On 7/8/14 1:02 PM, Ned Freed wrote:
>> The main issue I have whenever this sort of thing come up is that ABNF
>> is there to specify syntax, not semantics.  The ABNF in 4566 correctly
>> says that the syntax of an att-name is that it's a token.  The
>> specification itself -- the rest of it, beyond the ABNF -- is there to
>> tell us what values to expect there, what to do with them, and how to
>> define extensions.
>
> I could not agree more. As things edge away from syntax and closer to
> semantics, ABNF stops being the right tool for the job.
>
> For heaven's sake, ABNF can't even handle things like complex ordering or "at
> most two of" that really can be viewed as part of the syntax. And people expect
> it to handle even higher level constructs that overlap existing syntax in
> complex ways? Please.

I agree that some things are better not done in ABNF.

OTOH, designing syntaxes to support backward compatible extensions in 
the future, and then actually defining those extensions in the future, 
is a very common thing. Having some tools and/or processes for doing so 
would be a very good thing. And perhaps some additional syntax support 
would help. Or maybe not. I'm open minded.

SDP is pretty crufty - it is hard to imagine a more poorly designed 
syntax for the purposes it now serves. (It has been pushed vastly beyond 
its originally intended purposes.) So we are left trying to keep it 
limping along, and keep the specifications comprehensible, while 
retaining backward compatibility.

We can, in principle, change the way the syntax is specified to make it 
more consistent and understandable, as long as the revised version 
matches exactly the same set of inputs. And in fact that is what I hope 
we do, within limits. But what we do to the base spec must not render 
any of the existing extensions invalid.

SDP has many extensions. If I count right there are 51 different RFCs 
that define extension attribute values. (There are also lots of 
extensions to other parts of SDP.)

The *right* thing to do for SDP is to deprecate it and introduce a 
better structured alternative. This was already tried, many years ago, 
but failed. People worked on SDPng for a long time. It was XML-based, 
and I think it was pretty reasonable. It didn't fail because SDPng was 
bad, but simply because it wasn't SDP. SDP is embedded in SIP and H.323 
and RTSP, and the countless deployments of that. There was no plausible 
way, and no incentive, to accomplish the migration.

> Which is not to say these sort of things can't be handled cleanly. A good
> example is RFC 5322 section 3.6.
>
> But complex overlapped syntaxes? My response to that is, "Just say no!"
>
> There was a bunch of this junk in RFC 1341 and RFC 1521, the early versions of
> MIME. When I did the RFC 2045/2046 revision, I summarily deleted all of it. And
> AFAIK it has not been missed. Indeed, if the number and type of questions I
> received was any indication it has resulted in less confusion overall.
>
> This doesn't mean you can't specify higher level constraints on top of existing
> sytnax. You can, you just don't use ABNF to do it. A good example where this
> was done are the various extensions to Sieve. Sieve has a simple stable
> syntax specified in RFC 5228. Many extensions have been defined, most recently
> draft-ietf-appsawg-sieve-duplicate-09, but a different format is used to
> specify the syntactic constraints on extensions. The one in the most recent
> draft is:
>
> Usage: "duplicate" [":handle" <handle: string>]
>                        [":header" <header-name: string> /
>                            ":uniqueid" <value: string>]
>                        [":seconds" <timeout: number>] [":last"]

I'm not familiar with that. Is the syntax for the above *defined*, or is 
it left to the reader to figure out? I have used similar syntaxes in the 
past, in software documentation. It is good enough for an informal 
description, but I wouldn't use it for a standard without a formal 
definition of the grammar.

One of the approaches SDP uses for defining attributes looks a bit like 
this. But it leaves a lot to hand waving.

> There are also limited - very limited IMO - cases where overlapped syntaxes are
> OK. Although it's fairly rare, this sometimes happens with media type
> parameters. I don't have a problem with media type parameter syntax being
> specifed in ABNF. But we're talking about adding constraints to a single token,
> not a complex overlay.
>
> And finally, there are cases where fully extensible syntaxes make sense, at
> least in the context of previous design decisions. The obvious example of this
> is IMAP, where extensions add commands and specify their syntax when they do.
> This seems to be worked out OK. (Personally, I'm not entirely happy with the
> approach taken in the base specification to syntax specification, but it's very
> small beer compared to other issues in the IMAP space.)
>
>> That many specs enumerate all the tokens that are valid at the time of
>> their writing isn't really relevant to this, as I see it.  Personally,
>> I think we should stop doing that *unless* we want to define something
>> that intentionally has no extensibility.  To me, this makes sense:
>
>>     florb-value = "true" / "false"
>
>> ...while this does not:
>
>>     florb-value = "true" / "false" / florb-ext
>>     florb-ext = token
>
> I agree; this gets close to the line if edging over it.

I understand. But it is widely used because it is *easy*, while still 
leaving a path for future extension. This technique is used *widely* in 
the ABNF of SIP.

We aren't contemplating a bis for sip any time soon. If we were, then I 
would be looking for something better.

>> The first is clearly saying that *syntactically*, there are only two
>> things that can appear in a florb-value.  You can safely write a
>> parser that looks for those and throws a syntax error if it sees
>> anything else.
>
>> What on Earth is the second saying, syntactically?  I'd better write
>> my parser to parse it as a token.  I presume there's something else in
>> the text that tells me what to do with "true" and "false"
>> semantically, and that explains the extensibility.  What's the point
>> of having that in the *syntax*?

Yes, there typically is text describing the semantics of true and false, 
and saying that anything else that matches is to be ignored unless 
defined by an extension supported by the application doing the parsing. 
And also accompanied by IANA considerations that set up a registry 
florb-values.

>> This sort of thing is also fine, as I see it:
>
>>     florb-value = token ; must be a registered item, as
>>                         ; defined in Section 3.2.1

Yes, this works. It especially works in this case, if this is just an 
enumeration of values. It is far less workable if instead of "token" it 
is "byte-string", and individual values have complex sub-syntax.

An advantage of having the known alternatives shown in the abnf is that 
it is quick to look up the base alternatives - they show up right there, 
without need to consult the text. But you still have to consult the 
registry for other values.

I realize this is a weak argument. The strongest argument is that is is 
a technique that is widely used.

>> Here we're using a comment in the syntax to point the reader to the
>> section that gives the semantics and explains the valid value.
>> References are good.
>
>> Twisting syntax specification around to try to make it go beyond
>> syntax is not good.
>
>> Clearly, opinions differ on this... but there's mine.
>
> The "code" I've "run" in this space says pretty clearly that complex
> overlapped syntaxes cause more problems than they solve. And I really
> don't think adding intersection capabilities to ABNF is a solution.

Let me give you another part of this story.

When defining the syntax of an extension, it is highly desirable to 
reuse rules that are defined in the base specification. E.g., SDP has 
definitions of:

media
fmt
proto
port
unicast-address
FQDN
token
integer
...

To define extensions that are stylistically consistent with the rest of 
SDP you really want to reuse these.

In specifications we typically just say something like:

token = <from RFC4566>

(Or something even less formal.)

But if you are reviewing that specification and want to formally verify 
the ABNF of the extension syntax then you must pull the full abnf for 
4566 and merge it with the syntax you are verifying.

This is true whether the new definition is formally linked to the old 
one (via =/) or not.

	Thanks,
	Paul

_______________________________________________
abnf-discuss mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/abnf-discuss