IETF Logo Mail Archive

Re: [Last-Call] Last Call: <draft-koster-rep-06.txt> (Robots Exclusion Protocol) to Informational RFC

Mark Nottingham <mnot@mnot.net> Mon, 28 February 2022 21:26 UTC

Return-Path: <mnot@mnot.net>
X-Original-To: last-call@ietfa.amsl.com
Delivered-To: last-call@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CFB3E3A153E; Mon, 28 Feb 2022 13:26:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.11
X-Spam-Level:
X-Spam-Status: No, score=-2.11 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mnot.net header.b=CDDVDPGH; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=hElUSsgQ
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YhjeftiLWgvU; Mon, 28 Feb 2022 13:26:04 -0800 (PST)
Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA3843A1535; Mon, 28 Feb 2022 13:26:04 -0800 (PST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id BB4A6320091A; Mon, 28 Feb 2022 16:26:03 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 28 Feb 2022 16:26:04 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mnot.net; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; bh=83N3l/NfE55Pt9 8Z1C+bIFXXelOei7MEi/RFWOkiDw8=; b=CDDVDPGH6G628Oei/dW1QCmcUd8Hhz jL+j/7WBPjF9GivSegUpHVjfz83OUixoLnNUik9Eft1ndCCukQE0DRTRuj+/bhrW uox7IAOxmIJJvvu2DQ5zBK/Vc5v0EnB9zUtjYEQVlimhos8KPZGn32qkNw8qLI1Y NaDavWbLTOIYAmdIeS4XK4Utc9nK7dfMvTBlG1L/c3ISz6dKvFiLQHn5Z4TOp+gB RgQyVrm1TbHrIpEBU8w4sELYEuoWyePELQceNOZ5+SfwogNYv7zH1wKjTHAsxpAy lSjHMe1QS0FeGkt0qYQh6HtHRpLXJOHw6kPVLqj3Oc0t/eJhs+stB0TA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=83N3l/NfE55Pt98Z1C+bIFXXelOei7MEi/RFWOkiD w8=; b=hElUSsgQJ2ffEiINwgq1/14mepeKaTSxIBbMhhSgHIyo4eFISxSyf1gzy 38+x1SUf+5dri6S2Pl+cJRmXfTl9i9twMIlPB/5iqtjfHzCZwEnGSuhZgDOUW7mW 0zobXDx+3DeybiE/5DYDOP/5/UOstmO32rCkDoSmv+fOxYycHxU2ID81uJyjQSz1 qBeIjLvezrzP5mF61W3m0iUTZjNVgeam01NOUls889MqEib3k4HuKb3mmJ14iTPu i2Fh0kXg0b+lUHxwqXVG0HI2UkDz2dczE6tZTDUCADkh3LxYwDcRfOszZTDYooE5 IRjq8yM/dvTK1LkaHdbwwekmLh+mA==
X-ME-Sender: <xms:6j0dYhCJMX9k_X-lVUePLdhapfKgnLlknsddElOYnhIPXbY3ZZcz2w> <xme:6j0dYvjX8Sta9UQgvujnmeApNngi3UrNUNkSG-RKIRLHN6uDdhfQe0USKONibdJTZ DrAgj2aCR2HEl3kYw>
X-ME-Received: <xmr:6j0dYskcAh562qJUGowGzTLltfHwJ87uyklSFNfzIpWwpRdwSbwqDiFPtb_D7uM_QkKkgtkdO6GjMBmPa4S46WoVvGJwZ07YTXEM4VdmHY1_sbhyyZOjT9id>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddruddttddgudegkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurheptggguffhjgffgffkfhfvofesth hqmhdthhdtvdenucfhrhhomhepofgrrhhkucfpohhtthhinhhghhgrmhcuoehmnhhothes mhhnohhtrdhnvghtqeenucggtffrrghtthgvrhhnpeekuddvleejgeethfevkefhtdevke elveekfeegleduiefhudegvdeiuefftddthfenucffohhmrghinhepihgvthhfrdhorhhg pdhmnhhothdrnhgvthenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrih hlfhhrohhmpehmnhhothesmhhnohhtrdhnvght
X-ME-Proxy: <xmx:6j0dYrxD8siL1qPRJABbR7MRsQ2WYQS6QwhgXD86_ffXDkaVOC9qsw> <xmx:6j0dYmSDle2ohPBBPm_qsSGChENToBJQ68E9P0Ep_AtFoFzsFkSYHA> <xmx:6j0dYuZly9WlP8B8tAnYJGYkJjCifDItndjphu_ejskjKDnkphMTAw> <xmx:6z0dYsNBxZGnzbHs67ZRtWgxxlLgOyrS0eQODU24qtMPz-De9YxI1Q>
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 28 Feb 2022 16:26:01 -0500 (EST)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\))
From: Mark Nottingham <mnot@mnot.net>
In-Reply-To: <164606998882.28525.15371769982481176731@ietfa.amsl.com>
Date: Tue, 1 Mar 2022 08:25:58 +1100
Cc: Ted Hardie <ted.ietf@gmail.com>, draft-koster-rep@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <F83D9538-68F0-438E-92DD-AE3C5D036A7A@mnot.net>
References: <164606998882.28525.15371769982481176731@ietfa.amsl.com>
To: last-call@ietf.org
X-Mailer: Apple Mail (2.3693.60.0.1.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/last-call/Jn6EeVcbgWBmzBQjTFKaLIJqlZ0>
Subject: Re: [Last-Call] Last Call: <draft-koster-rep-06.txt> (Robots Exclusion Protocol) to Informational RFC
X-BeenThere: last-call@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF Last Calls <last-call.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/last-call>, <mailto:last-call-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/last-call/>
List-Post: <mailto:last-call@ietf.org>
List-Help: <mailto:last-call-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/last-call>, <mailto:last-call-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Feb 2022 21:26:10 -0000

Thanks to the authors for doing this work; having it documented in an IETF-stream RFC would indeed be a good thing.

I'd be *much* more comfortable doing so if we had public indication from other consumers of this format (especially: competing search engines) that the definition is correct and interoperable with their implementations.

Cheers,


> On 1 Mar 2022, at 4:39 am, The IESG <iesg-secretary@ietf.org> wrote:
> 
> 
> The IESG has received a request from an individual submitter to consider the
> following document: - 'Robots Exclusion Protocol'
>  <draft-koster-rep-06.txt> as Informational RFC
> 
> The IESG plans to make a decision in the next few weeks, and solicits final
> comments on this action. Please send substantive comments to the
> last-call@ietf.org mailing lists by 2022-04-07. Exceptionally, comments may
> be sent to iesg@ietf.org instead. In either case, please retain the beginning
> of the Subject line to allow automated sorting.
> 
> Abstract
> 
> 
>   This document specifies and extends the "Robots Exclusion Protocol"
>   method originally defined by Martijn Koster in 1996 for service
>   owners to control how content served by their services may be
>   accessed, if at all, by automatic clients known as crawlers.
> 
> 
> 
> 
> The file can be obtained via
> https://datatracker.ietf.org/doc/draft-koster-rep/
> 
> 
> 
> No IPR declarations have been submitted directly on this I-D.
> 
> 
> 
> 
> 
> _______________________________________________
> IETF-Announce mailing list
> IETF-Announce@ietf.org
> https://www.ietf.org/mailman/listinfo/ietf-announce

--
Mark Nottingham   https://www.mnot.net/

v2.8.7 | Report a Bug | By Email