[rfc-i] v3imp #8 Fragment tagging on sourcecode

[dev+ietf at seantek.com (Sean Leonard)]

On 1/28/2015 6:56 AM, Paul Kyzivat wrote:
> On 1/28/15 12:16 AM, Sean Leonard wrote:
>> Overall I still stand by my proposition that the RFC is the module for
>> ABNF purposes. Honestly it just makes things a lot simpler. To the
>> extent that you need to split things inside the RFC, you can refer to
>> specific sections. Specific comments below.
>>
>> With regard to import rules, I can concur with Julian's comment that
>> ABNF should not be extended.
>>
>> Instead, I propose the following *informal* definition, which is based
>> on Section 2.7 in RFC 7230 (and is repeated by Julian below, in RFC 
>> 7231):
>>
>> localrule = <foreignrule, see [RFCXXXX], Section Y.Z>
>
> Claiming that this is not an extension is a distinction without a 
> difference. If it is well defined enough that a tool can follow it to 
> obtain the needed definitions then it *is* an extension. We are only 
> arguing about the syntax. (And I haven't proposed a syntax.)

I believe that proposing concrete syntaxes and examples helps...so 
please, propose a syntax. :)

But as I said, I am no longer proposing an extension or a syntax change 
to RFC 5234. Common phraseology is more of a convenience for tools. For 
example, it is common in RFCs to phrase inter-RFC references such as:

"Use the LOOFAH production in Section 4.5 of [RFC9989]."

There is no standard (that I am aware of) that requires references to 
sections to be phrased "Section X.Y of [RFCZZZZ]." However the tool on 
tools.ietf.org/html is smart enough to recognize this phraseology and 
generate <a href=""> linkage with fragments to other pages in 
tools.ietf.org/html. Thus it is a "convention", not a formal syntax.

>
> This also begs the question of exactly what is imported. As I 
> mentioned later, there are options, and each has an impact. So that 
> also needs to be specified for tools to do the right thing.

Or if we obviate (eliminate) the need to "import" anything, we are good 
and have not complicated anything.
>> All an ABNF validator needs to do is make sure that all rules are
>> comprised of valid ABNF primitives, or other rules that decompose into
>> valid ABNF primitives. ABNF primitives are:
>> 1. literals, such as %d13 and %x0D (which, incidentally, are equivalent)
>> 2. rules assumed to exist (i.e., RFC 5234 Appendix B)
>> 1. rules defined by <>
>
> I disagree. It is also important to check that all references to rules 
> are satisfied, and that there are no conflicts between rules.

I demur.

As long as rule names aren't "imported" into the local (intra-document) 
namespace, it's a non-issue.

I don't know what you mean by "it is [also] important". Of course for 
the editorial process, you want to be sure that:
---
URI = <URI, see Appendix A of [RFC3986]>
---
is correct. But the same would be said of the in-specification text:
---
The ABNF in Figure X below uses the URI production in Appendix A of 
[RFC3986].
---
I.e., there is no difference in the editorial issue. If you want 
something more automated than human eyeballs, please propose a syntax.

>
>> Such a validator is quite easy to program, and doesn't need to import
>> anything from other RFCs.
>
> Again I disagree. As I've noted before, in RAI there is a custom of 
> having drafts that extend the ABNF of other drafts. I've used bap to 
> verify ABNF numerous times, and I've had to do a lot of manual 
> preprocessing and importing/merging to thoroughly verify that all is 
> well. That is all error prone, and inclines people to not bother with 
> actually doing the verification.

Perhaps some concrete examples would help for the benefit of this list, 
for those not familiar with RAI drafts.

However suppose that you extend a rule, such as:
---
ruleexample    = 5ALPHA
ruleexample    /= 5DIGIT
---

Is there much difference for ABNF validation purposes between the above, 
and the below?
---
ruleexample    = <ruleexample from draft-seantek-rai-whatever-03>
ruleexample    /= 5DIGIT
---

Nope...both will validate just fine per my 3 proposed primitive categories.

>
>> On 1/27/2015 12:53 PM, Paul Kyzivat wrote:
>>> On 1/25/15 3:57 AM, Julian Reschke wrote:
>>> Neither. See how it works in RFC 7230 and RFC 7231. {localrule =
>>> <foreignrule, ...>}. The rule name of the foreign rule is irrelevant
>>> since a local rule is defined with the standard syntax.
>
> How do you assume this "import" to work? It could be equivalent to:
>
> localrule = foreignrule
> <insert text of foreignrule>
>
> OR
>
> localrule = <insert right hand side of foreignrule>
>
> Either way, if the right hand side of the foreign rule contains 
> references to other rules there is more to do.

I think you are overthinking my proposal. Again, for ABNF validation 
purposes, <> is primitive. No importing or substitution needs to be done.

If you're talking about ABNF compiling--as in actually using the rules, 
I don't see anything wrong either. If a foreign rule contains references 
to other rules, it's appropriate to "compile" the foreign rule into 
primitives, so that the foreign rule can be applied as the localrule 
without "references".

>
> One possibility would be to import some or all of the referenced 
> foreign ABNF text but rewrite it, replacing all the rulenames with 
> obfuscated names that don't conflict. But that is complex, and has its 
> own consequences.

Or as I said, don't import rulenames at all...just replace all rulenames 
with primitives. I still submit that this is a job of an ABNF compiler 
rather than an ABNF validator.

When I say "ABNF validator" I mean a process that verifies that the ABNF 
syntax is consistent with itself. This definition *excludes* a process 
that compiles the ABNF syntax and verifies arbitrary inputs as 
conforming to particular ABNF productions.

>
> When I have verified in the past, I have either imported *all* the 
> ABNF from the referenced document, or else I have imported rule 
> definitions one by one until I no longer have unsatisfied references. 
> Or, in the case of core rules, I have simply imported all the core 
> rules and nothing else from 5234.

We may be using different definitions of "verify" ("validate"). If you 
treat <...> as primitive, you don't need to do that.
> Agreed, there is no conflict *there*.
>
> The question comes for those who want to reuse the core rules. Must 
> users of those rules also refrain from using the names of rules in 
> section 4 for their own purpose?

Nope. Again, the core rules are special--they should be taken for 
granted as a condition of using ABNF, so it shouldn't be necessary to 
refer to them via RFC 5234.

Sean