I-D Action: draft-koster-rep-12.txt

internet-drafts@ietf.org Wed, 06 July 2022 20:32 UTC

MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: internet-drafts@ietf.org
To: i-d-announce@ietf.org
Subject: I-D Action: draft-koster-rep-12.txt
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <165713952770.59423.13322127253331280904@ietfa.amsl.com>
Date: Wed, 06 Jul 2022 13:32:07 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/i-d-announce/l855mX6AkRahHQv-5KJGo9D9eeo>

A New Internet-Draft is available from the on-line Internet-Drafts directories.


        Title           : Robots Exclusion Protocol
        Authors         : Martijn Koster
                          Gary Illyes
                          Henner Zeller
                          Lizzi Sassman
  Filename        : draft-koster-rep-12.txt
  Pages           : 14
  Date            : 2022-07-06

Abstract:
   This document specifies and extends the "Robots Exclusion Protocol"
   method originally defined by Martijn Koster in 1996 for service
   owners to control how content served by their services may be
   accessed, if at all, by automatic clients known as crawlers.
   Specifically, it adds definition language for the protocol and
   instructions for handling errors and caching.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-koster-rep/

There is also an htmlized version available at:
https://datatracker.ietf.org/doc/html/draft-koster-rep-12

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-koster-rep-12


Internet-Drafts are also available by rsync at rsync.ietf.org::internet-drafts

I-D Action: draft-koster-rep-12.txt internet-drafts