Re: [Last-Call] Last Call: <draft-koster-rep-06.txt> (Robots Exclusion Protocol) to Informational RFC

[S Moonesamy <[e-mail]>]

Hi Ted,
At 01:18 AM 08-03-2022, Ted Hardie wrote:
>On the more general topic of why this has the "no derivatives" 
>clause,  I understand your reluctance, but I think this is a case 
>where the combination is valid.  First, it's important to note that 
>the specification was brought to the IETF for substantive review, to 
>make sure that the elements it uses (like ABNF) were being used in 
>the right way and to eliminate any possibility of ambiguity.  From 
>my perspective, that's been very useful and it would not have 
>occurred to the same extent had this gone directly to the ISE.

I read that the ISE did not have the expertise to review elements 
such as ABNF.  It took me a few minutes to find the ABNF is Section 2 
of RFC 8905.

>However, this spec reflects operations which have been 
>stable/backwards compatible for a very long time.  Given that, it is 
>important to the community which deploys this that it be fairly 
>difficult to amend.  One way to achieve that would have been to make 
>this standards track; that would require standards action to update 
>or obsolete it later.  When we discussed that back at the beginning 
>of this process, though, it was pretty clear that some folks would 
>use the working group discussion around that to try to insert 
>functionality that would result in breaking changes.  While it would 
>have been kind of unlikely for any of those to win out against the 
>need for maintaining interoperability, the result would have been a 
>pretty big increase in the amount of effort needed to get this published.

The was an email on 24 February with the following paragraph: "The 
IETF prides itself on its open process and transparent communication, 
with one of our core principles being our commitment to making all 
materials related to our standards process and other activities 
publicly available."  I was unable to find the discussion which 
occurred at the beginning of the process.

>Another option for getting an archival spec with a high bar for 
>change was this one:  an IETF informational with a no-derivatives 
>clause. That gave the full benefit of IETF review and made the bar 
>for amendment high enough to allay the concerns of the original 
>author and the relevant community.  It had this clause when Adam 
>agreed to sponsor it and it has had it in every iteration since, so 
>I thought this was well understood.  As shepherd, my apologies if it was not.

 From what I understand, the "IETF informational" imprimatur isn't 
about creating archival specifications.

>But, absent that, I think this kind of document is why BCP78 permits 
>this combination: documents which need and have received significant 
>IETF review but which also have a significant external community for 
>whom the usual clauses result in a risk of inappropriate later 
>amendments. To put this slightly differently, I think you'll see 
>that this falls under the logic in RFC 5378, Section 3, in the 
>penultimate paragraph.
>
>If you and the broader community prefer the standards track 
>approach, now would be a good time to let the sponsoring AD know.

The following sentence is from RFC 5378: "The IETF has historically 
encouraged organizations to publish details of their technologies, 
even when the technologies are proprietary, because understanding how 
existing technology is being used helps when developing new 
technology."  There is information on the web to understand how 
robots.txt is being used.  Why is an IETF RFC with a "no derivative" 
clause useful in this context?

Regards,
S. Moonesamy