IETF Logo Mail Archive

Re: Draft submission: Robots Exclusion Protocol

Mark Nottingham <mnot@mnot.net> Fri, 18 June 2021 04:54 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 19AA73A3C49 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 17 Jun 2021 21:54:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.749
X-Spam-Level:
X-Spam-Status: No, score=-7.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mnot.net header.b=fHXIKI3/; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Gw0zjEKi
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9mBpYAaFC07O for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 17 Jun 2021 21:54:34 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 962FE3A3C47 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 17 Jun 2021 21:54:33 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1lu6SB-0002tp-UI for ietf-http-wg-dist@listhub.w3.org; Fri, 18 Jun 2021 04:50:08 +0000
Resent-Date: Fri, 18 Jun 2021 04:50:03 +0000
Resent-Message-Id: <E1lu6SB-0002tp-UI@lyra.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <mnot@mnot.net>) id 1lu6Qh-0002fm-Vv for ietf-http-wg@listhub.w3.org; Fri, 18 Jun 2021 04:48:39 +0000
Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by mimas.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <mnot@mnot.net>) id 1lu6QY-0002VJ-0V for ietf-http-wg@w3.org; Fri, 18 Jun 2021 04:48:27 +0000
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 3887EFCA; Fri, 18 Jun 2021 00:48:06 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 18 Jun 2021 00:48:06 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mnot.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm3; bh=e 8UY+dxSsq6Ms/qulk9BQKaZblpIfI6pHII63ZgTC08=; b=fHXIKI3/uyrxSDTQ/ EZ8LuJQx7FB3Sl7gWjYHf6lcVdJZ5JoP8c6hF4ioKNbo1sEOF6GaADQ0qMd8htco bggV9Lm+ieTcAUWgo/fc3PotjrFO1NIEZCyICa73df/OisttxTDOEU6/cPRPnwIf P9JVGmX1fa5PKHpM9+gCVvjnF0n5EOIMtv03lz3IQ0FHj7uuaoSofB3NYDNymch3 HTItFOg3ZoDlVxsKQKifAbt7gdrDiqFQ2Bcp6Jha4VDrgSUrGNApdgCayRCHUayf J3muIz2x7sRRQTfxFGcyhBOZgsv1HYY5xQ8nxnNlx2uEbWQbiD4hlinfMnUNgksN 1zN/g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=e8UY+dxSsq6Ms/qulk9BQKaZblpIfI6pHII63ZgTC 08=; b=Gw0zjEKihj2jQTdn+GUFXevU5fu5X3+ztkLV51FuMqY2ZlXtnRiACd17O jcusYRssPCaJp+RIxYmJIi6qX0g71ep9viQdsBvYhFo8Ov95yNe+10dF08GVhvEj AJAFNvmQOv6IsLNwJbHqTtnhiZeTcjhjanw8g2WnzBpMEMw4LKcj59x0J6A4p92S FKJOSEp4GV5gFUvMu2mPafNuDtchvc/F+gJ8j3PGwzWdYEwFB7KKGvxVqJzimvqq S92qFd+VMTtxOF+bsBHukWI90Pehi+rCnSFyLdNI7gsjOWN2BF7IDVKQG/p6G7cT Fq3jMgO3tAA6Ftjcs/CPvryP0uohg==
X-ME-Sender: <xms:hSXMYAn1asendJ1OR_jIBaBo93hlK09QmDu5Bs8cS4g9jb1L0S5Bhw> <xme:hSXMYP2G8vpTwVTl7oXzg7MG11vZFjq9W9d7k1ZYd-DKjpGCwj8cY8C7r0pQxpqca hdNMMYBc4ocNTvsRw>
X-ME-Received: <xmr:hSXMYOofJ43KGBjcPPBhel6r0Orn4z-qXY4eDqPj8AhFGZyM5JG_Rxaaafw-8uZFhENPlH673vaV0aOpGGNlTC1XJ0caysy_HFyc6ThMPaKoawT-DV6F_Juc>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeefvddgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomhepofgrrhhk ucfpohhtthhinhhghhgrmhcuoehmnhhothesmhhnohhtrdhnvghtqeenucggtffrrghtth gvrhhnpeekuddvleejgeethfevkefhtdevkeelveekfeegleduiefhudegvdeiuefftddt hfenucffohhmrghinhepihgvthhfrdhorhhgpdhmnhhothdrnhgvthenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmnhhothesmhhnohhtrdhn vght
X-ME-Proxy: <xmx:hSXMYMk-8NyDVrypVlpWs4EkhFvvdANVxsB9r1i6GZMO4Q0UwILRCw> <xmx:hSXMYO0VQ1Zsr8H86MR5qUiW8mNn3LP9AK7Gimh1WTUcdx1o89zB1g> <xmx:hSXMYDvgpT3K-8A_IqJey5retS2lejLLF3bLfwmaAEQ_EJZJlFLhqg> <xmx:hSXMYDzy1eu_wAnsyv-69RrI-ZTsziFOe7iGO6HpYlcttAbAyEmm0w>
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 18 Jun 2021 00:48:03 -0400 (EDT)
Content-Type: text/plain; charset="us-ascii"
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\))
From: Mark Nottingham <mnot@mnot.net>
In-Reply-To: <CADTQi=dnWwgFC65KEwj7WqHjvm=-ZEAv6iRODGUq=_LM+TWyDg@mail.gmail.com>
Date: Fri, 18 Jun 2021 14:48:01 +1000
Cc: ietf-http-wg@w3.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <C7E09C19-7CD1-41E5-8935-0C0BDFCD4266@mnot.net>
References: <CADTQi=dnWwgFC65KEwj7WqHjvm=-ZEAv6iRODGUq=_LM+TWyDg@mail.gmail.com>
To: Gary Illyes <garyillyes@google.com>
X-Mailer: Apple Mail (2.3654.100.0.2.22)
Received-SPF: pass client-ip=64.147.123.20; envelope-from=mnot@mnot.net; helo=wout4-smtp.messagingengine.com
X-W3C-Hub-DKIM-Status: validation passed: (address=mnot@mnot.net domain=mnot.net), signature is good
X-W3C-Hub-DKIM-Status: validation passed: (address=mnot@mnot.net domain=messagingengine.com), signature is good
X-W3C-Hub-Spam-Status: No, score=-9.8
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1lu6QY-0002VJ-0V ea1cf7121297d2f5ca1607286858e7aa
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft submission: Robots Exclusion Protocol
Archived-At: <https://www.w3.org/mid/C7E09C19-7CD1-41E5-8935-0C0BDFCD4266@mnot.net>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/38919
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Hi Gary,

This didn't get much discussion when you sent the initial e-mail, but I see you've been making steady progress on the draft.

Have you considered taking it to DISPATCH? That's probably the most effective way to determine where this could live in the IETF. Having a definitive home for it would help to assure that it incorporates input across the community and reflects consensus.

See:
  https://trac.ietf.org/trac/dispatch/wiki

Cheers,



> On 8 Jul 2019, at 6:46 am, Gary Illyes <garyillyes@google.com> wrote:
> 
> Hi HTTP WG,
> 
> As you may have seen, a group has gotten together with Martijn Koster to create an internet-draft which restates the Robots Exclusion Protocol using updated ABNF and prescriptions.  The current draft is (draft-koster-rep).  If you get a chance, we'd appreciate your confirming that it matches your understanding of current practice and that the specification is clear even for edge cases.  You can send mail to me to forward on to the author team or address it to the emails listed in the draft.  
> 
> Thanks for your help,
> Gary Illyes
> Google Switzerland
> 
> https://datatracker.ietf.org/doc/draft-koster-rep/



--
Mark Nottingham   https://www.mnot.net/

v2.23.0 | Report a Bug | By Email