Protocol Action: 'Robots Exclusion Protocol' to Proposed Standard (draft-koster-rep-12.txt)
The IESG <iesg-secretary@ietf.org> Thu, 14 July 2022 18:30 UTC
Return-Path: <iesg-secretary@ietf.org>
X-Original-To: ietf-announce@ietf.org
Delivered-To: ietf-announce@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id B4738C15790B; Thu, 14 Jul 2022 11:30:31 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: The IESG <iesg-secretary@ietf.org>
To: IETF-Announce <ietf-announce@ietf.org>
Subject: Protocol Action: 'Robots Exclusion Protocol' to Proposed Standard (draft-koster-rep-12.txt)
X-Test-IDTracker: no
X-IETF-IDTracker: 8.6.0
Auto-Submitted: auto-generated
Precedence: bulk
Cc: Ted Hardie <ted.ietf@gmail.com>, The IESG <iesg@ietf.org>, draft-koster-rep@ietf.org, rfc-editor@rfc-editor.org, superuser@gmail.com, ted.ietf@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-ID: <165782343172.35841.3539236635295035164@ietfa.amsl.com>
Date: Thu, 14 Jul 2022 11:30:31 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf-announce/3TVXCKSoUmeHhYPdD0GYBpEuqtM>
X-BeenThere: ietf-announce@ietf.org
X-Mailman-Version: 2.1.39
List-Id: "IETF announcement list. No discussions." <ietf-announce.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-announce>, <mailto:ietf-announce-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf-announce/>
List-Post: <mailto:ietf-announce@ietf.org>
List-Help: <mailto:ietf-announce-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-announce>, <mailto:ietf-announce-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jul 2022 18:30:31 -0000
The IESG has approved the following document: - 'Robots Exclusion Protocol' (draft-koster-rep-12.txt) as Proposed Standard This document has been reviewed in the IETF but is not the product of an IETF Working Group. The IESG contact person is Murray Kucherawy. A URL of this Internet Draft is: https://datatracker.ietf.org/doc/draft-koster-rep/ Technical Summary This document specifies and extends the "Robots Exclusion Protocol" <http://www.robotstxt.org/> method originally defined by Martijn Koster in 1996 for service owners to control how content served by their services may be accessed, if at all, by automatic clients known as crawlers. Working Group Summary This is an AD-sponsored document. It went through two Last Calls. The first had it at Informational with a "no derivative works" clause, and the community objected strongly to the latter and expressed a clear preference for the Standards Track, so the second Last Call was done after those changes were made. There is general support for publication. Suggested changes were duly considered, so that feedback has been addressed. There has been some expression of dismay that this didn't go via a working group. Document Quality This document describes a protocol that has already been widely implemented. Some large web property operators have indicated their support for the protocol as documented. Personnel Ted Hardie is the document shepherd. Murray Kucherawy is the responsible Area Director.