Re: [Last-Call] Last Call: <draft-koster-rep-06.txt> (Robots Exclusion Protocol) to Informational RFC

[Gary Illyes <[e-mail]>]

All, we had this boilerplate since the very first draft, so I'm rather
amused it only surfaced now as an issue. We thought we need the
current "no modifications, no derivatives" boilerplate so consumers
and users of the Robots Exclusion Protocol (REP), in particular
webmasters and search engines, don't end up with
backwards-incompatible changes in the draft.  As Martijn
mentioned, the REP has been around for a very long time, over
450 million hosts rely on it in addition to search engines, so we
feel rather protective about it.

With all that said, it's clear that the current boilerplate is a no-go
and, based on some sidebar chats, our worries perhaps unfounded,
so we can switch the status section to the "IETF Informational w/
consensus" one
(https://www.rfc-editor.org/materials/status-memos.txt).

Additionally, addressing Mark's comment about an ACK from other
consumers of the REP, I'll put out a public request on an official
Google social media account inviting other search engines to reply
here. We did check in with them before submitting the draft as well
as when we made notable changes (v5 -> v6), but I agree that the
other consumers of the REP should also at least acknowledge the
publication of this draft, considering how important it is in general.

Let me know if I missed some comments that need my attention.

Gary

On Fri, Mar 11, 2022 at 11:23 AM Martijn Koster <[e-mail]>
wrote:

>
>
> > On 9 Mar 2022, at 14:15, Salz, Rich <[e-mail]> wrote:
> >
> >> I don't like to guess,
> >
> > Let's not guess, let's ask the authors directly.
> >
> >> Can someone – probably one of the authors, not Ted – why they feel it
> is necessary to not allow derivative works?
> >
> >> In the past there have been many RFC’s that documented existing
> cryptographic algorithms: Blake2, ChaCha/Poly, etc. None of them say no
> derivatives.
> >
> > Why do you want no-derived-changes?  Given that the IETF almost never
> does this, I would expect this to be a hard requirement to justify.
>
> Gary, why was this added? Having looked at the discussion in RFC3667
> sections 5.2 and 7.3 it seems that this is appropriate for re-publishing
> documents from other standard bodies, and to protect proprietary
> technologies. The robots.txt is based on a long-standing industry practice,
> defined by a historical specification and de-facto (diverging)
> implementations, but not a standard body. And the proprietary technology
> protection doesn’t apply either.
> So I agree it should not be there.
>
> — Martijn
>
>