[ai-control] Re: New draft: draft-kelley-aipref-training-preference-signal-00
Kevin Kelley <kelleyk-ietf@kelleyk.net> Thu, 12 November 2026 01:07 UTC
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com [IPv6:2a00:1450:4864:20::22d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by mx.ietf.org (Postfix) with ESMTPS id 0E1004B for <ai-control@ietf.org>; Thu, 12 Nov 2026 01:07:56 +0000 (UTC)
Authentication-Results: mx.ietf.org; dkim=pass header.d=kelleyk.net header.s=google header.b=Eezv6rJ5; spf=pass (mx.ietf.org: domain of kelleyk-ietf@kelleyk.net designates 2a00:1450:4864:20::22d as permitted sender) smtp.mailfrom=kelleyk-ietf@kelleyk.net; dmarc=none; arc=pass ("google.com:s=arc-20260327:i=1")
Received: by mail-lj1-x22d.google.com with SMTP id 38308e7fff4ca-3ae1fd4252fso923171fa.0 for <ai-control@ietf.org>; Wed, 11 Nov 2026 17:07:55 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1794445669; cv=none; d=google.com; s=arc-20260327; b=TI+YNNiw3Wr5b56Za+VipgMId8odnCwr5F+NwRBU8YkfftnpZ9ATF9DxdadovybCMd SZ9frS/Rq1aM6aQEu3Dg3rhGE3f62mLcV8q7RHICn6LaBKe8K52oR9wc9QlsQrWsJCv/ HoTJ1H0wiPtCJFEGf+tjszI4nozZI0kaXyBPGL0k6lynMOCUbmBTKM728aBLpz0s/zDZ jEqyH54lIzhUsY3j4nsUtPCs4qo11aR8WtBmiiYgFEF4V3mvzPZeuqUe5qbvQmCAtTro /toElwsbVFhz8bdE5ZCV/8loNUANiM66c5u7o3xTauWcYB2sO7pPqbAj+U0keBOwmYZx MMqw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=fdxtXzsvZptvL+d01Nuz8I4TkQ31i+Vma3yBaH5RTLM=; fh=4VFjxvGiF8SPWVtT4XYPe2eCYl0txx5bxzQzmR3cLiU=; b=P8UBMId0XbZRCnHuF8y2vZ799gK+vrEwlVRXnAwyIUrMRSDYzqgao2w2uodRLvoJXa zbnI3itEcfd4k1yzt0PjRqG2URYw74Qy6Dr+quPoLiM7lY6wmGF7qpNlgcoymQJ4OAIQ bRsRR1fCvU8EWkrlUPb7pKSE3g8T2XyxTXqPf34k2g24ccw5GURCVGdzk1F/dtNu1xCd vk4FRnCOM0IH9ibBV75rFROPAExPWPkx+JAfgcktGjs9IooMH6iuZjAS/CkUhWyuAXdH Yq8oT3qdjvh2CSWAi6z7EQiF99XUhAOARpi/1TFPAONjYZLq7+DXHvLbokLk2TT1bHJ3 djaw==; darn=ietf.org
ARC-Authentication-Results: i=1; mx.google.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kelleyk.net; s=google; t=1794445669; x=1795050469; darn=ietf.org; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:from:to:cc:subject:date:message-id:reply-to :content-type; bh=fdxtXzsvZptvL+d01Nuz8I4TkQ31i+Vma3yBaH5RTLM=; b=Eezv6rJ5OdI8I4nwoSttn1gjakYd/8G4t4I9F7i2um5sWEQfQE59GaxnPHSeKuSPEy rvPIqNxiJFqyXkka3RK/6FLwGUeeekZrYgeKvQG6bEjVr778m3NNSnrGjhnNkQFrz/11 TdsLt3xOgGr+/BO01RwjrqnFW2mzVKzEePSYA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794445669; x=1795050469; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to:content-type; bh=fdxtXzsvZptvL+d01Nuz8I4TkQ31i+Vma3yBaH5RTLM=; b=J+gfB1Y61FR4hieI/Bv7QC7m7hI43wYDLHje0wbNvxyuqRS6EB7W9U8u51rX2Hxvij AE9qAtENkKoN2NVWZSNgafVQOKNhWJ4g6jA+jEePez5PH6GxFzQu7ToEk9OZcAecicr7 nAjTR6kfbHUUCf93KTWYxA6GebTFkDp8wWlAvUh9U3IQUEs4QlH/UZAScXlr3mze/G7o mOcwC/3gxNxntWxRXHC0cVfgI/4S8ZOc+5B2gGp3XYp+UBPA3ruVBOJsqy3SR/fpkfEk o67QqHcvdZou9zdX39chlDeT7aEFDqLl31gVM7UHR7TcdNYqdefF7ExGYylw+opNkdoI 18Mg==
X-Gm-Message-State: APuU4Z8KnU0ZNbXIcj4qdNs5eIQdP+hsRgmbV1clLOTz33WXiT/TE7Pi ySdS+RkPYNPsFUsIItJAfaSLyhPxKpR1W+KoOfvlMpWHOT+YKWDOyiyqV0A36I/PpKd/kDnxIyn 5vUmMpYSozw/OlVArAPTWWBTrDvk5GAT8BmD01AhE8EmQ1zemxzHcNExn8g==
X-Gm-Gg: AYBFou0IoDI9vI+Z1NcXZnBhAMXWu43aJMbX0mNaxy1R4iJuNU8zM24lgHSgxECpWRn WeEDdYt69cMryaKS78uEyHkcsYtEovXOI6B22bda7Q9FQDknXjQwVLRDi+D9CWS/VynUPxTCOa8 TSqTBRwZk61jkWErJcmyPQO9aCH/YJN2ADOoru/JV51X68erbYF5K3AY+YypekjdJ7wC0FQ1087 WuN2ko7Rh+jJK/fXcMJfeVmaS1dFKPb0aj2mL+ofQPcbqR+pz9WYeEJl/myiEx6iaLh9stCTySR vaspYmZugUytLB76L7tk2ELqa90BmbVMO4kBuH8yNaaSK17wlwoVkiSarebsMbz3+k6dZADbV4o hAVKZgak/eKFVZ8Kbc1F41hJEeJrOJZid8m40HLzF7MYKwnaagfU8udNIF+3UmLmpwvSnJw0UdL uUYlRCDYgy3mCLNdE9CSQ=
X-Received: by 2002:a2e:8a94:0:b0:3ac:e24f:aa0d with SMTP id 38308e7fff4ca-3ae126698f4mr8279851fa.7.1794445668323; Wed, 11 Nov 2026 17:07:48 -0800 (PST)
MIME-Version: 1.0
References: <CAGfqOAY_t6UmzZw+nYzTfRVhyHuGb7RyEwmcwetwAF01f4ONbA@mail.gmail.com> <CAOC05+J7Zs97jXaAvwU_a1LimYMfTkExULWADY3F=z1Dnip=GA@mail.gmail.com> <CAN99zG35KyYTFVeDYD1UW6fWkUn+ojr3zxRn6aR1DhD07wsURA@mail.gmail.com> <CAGfqOAbd0PmA9VR2OYXVDGFH_iraA94gFm5u+pDg08uFGtR9GQ@mail.gmail.com>
In-Reply-To: <CAGfqOAbd0PmA9VR2OYXVDGFH_iraA94gFm5u+pDg08uFGtR9GQ@mail.gmail.com>
From: Kevin Kelley <kelleyk-ietf@kelleyk.net>
Date: Wed, 11 Nov 2026 17:07:36 -0800
X-Gm-Features: AclHuK-3KpFnW67jrCBYxC5vpoMiELuAsqWFTI8Qa42uyqWTqxWvS_53IUEOKoM
Message-ID: <CAGfqOAZwS2bO0_iKaHG7Q42KfZCpb7+ukj_FLZ2m5xHoOHeWDg@mail.gmail.com>
To: Alex Gibson <agibson@newscorp.com>
Content-Type: multipart/alternative; boundary="000000000000a11f2e066009f0f4"
X-Spamd-Bar: ----------------
Message-ID-Hash: IRUDMKH7JW2GZ67ANFL25BSVM6ERQUIQ
X-Message-ID-Hash: IRUDMKH7JW2GZ67ANFL25BSVM6ERQUIQ
X-MailFrom: kelleyk-ietf@kelleyk.net
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
CC: ai-control@ietf.org
X-Mailman-Version: 3.3.10
Precedence: list
Subject: [ai-control] Re: New draft: draft-kelley-aipref-training-preference-signal-00
List-Id: AI Control <ai-control.ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/ai-control/Nekq7BRALetH3CllUwDXIEwM28c>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ai-control>
List-Help: <mailto:ai-control-request@ietf.org?subject=help>
List-Owner: <mailto:ai-control-owner@ietf.org>
List-Post: <mailto:ai-control@ietf.org>
List-Subscribe: <mailto:ai-control-join@ietf.org>
List-Unsubscribe: <mailto:ai-control-leave@ietf.org>
Mark, Alissa, Nate, My draft was intended to be a self-contained description of what I was announcing, and not a wholesale replacement for any of the Working Group’s efforts. It doesn't contribute anything particularly novel on the vocabulary side, since the "AI training" definition it uses is exactly the one from the Working Group's current draft. I included that term and not others only because that definition was the first to stabilize. We’ve done some refactoring, added some clarifying examples, and so on, but there hasn’t been much discussion about whether we’re drawing a line in a useful place or about the meat of the definition in quite a while. In contrast, we set aside discussion of inference-time use to make progress on training, and my sense is that there are still more philosophical conversations to be had there. I have a lot of respect for the IETF process and for the work we’ve put into finding common ground in AIPREF. The multilateral nature of the group is what makes the work we’re doing together practically useful for everyone involved (in contrast to the variety of unilateral attempts at addressing similar problems); and I was concerned that folks would not appreciate it if I announced signals based on less stable definitions. If I read that incorrectly, and you would have preferred that I had included more of the vocabulary, that would be useful feedback, and I hope you feel comfortable sharing it, either here or in private. Like I said in my first message in this thread, I’m committed to continuing to participate in the Working Group, and to working with everyone to try to describe, and find mutually workable definitions for, other interesting categories of use. The draft does contribute on the attachment side, which was necessary because that work has been on hold in the Working Group while we’ve been focused on the vocab. As much as the history of robots.txt makes working with it delicate (which I hear some people pronounce “frustrating”), I’ve always seen attachment as a much more mechanical part of the problem. I landed in the place I described in my individual draft while navigating a long list of interoperability issues. That work *is* something that I wanted to offer to the Working Group, though more because I hope it lets us make progress as we sort out vocabulary issues than because I’m attached to the specifics. On the concerns around the lack of intentionally extensible syntax: my initial drafts were actually written that way! I pulled that out of the version that I submitted as -00 based on strong off-list feedback suggesting that it made the draft harder to follow. I’ve posted a draft that updates RFC 9309 to add namespaced labels, and will submit it once the Datatracker reopens. You can read that here, though it’s purely focused on how to get robots.txt to carry distinct signals: https://kelleyk.github.io/draft-kelley-rep-namespaces/draft-kelley-rep-namespaces-00/draft-kelley-rep-namespaces.html Mark, I’ll also open a PR against the attachment draft that proposes defining a “usage” namespace and mapping the entire vocabulary to labels in it. Happy to answer questions here, or you are all welcome to reach out directly or pull me aside in SF if you'd prefer. Cheers, -K On Wed, Nov 11, 2026 at 4:38 PM Kevin Kelley <kelleyk-ietf@kelleyk.net> wrote: > Alex, > > I see the draft I submitted as describing a checkpoint in the Working > Group's work, so it felt important to me to acknowledge everyone who came > together to do that work rather than submitting a draft with only my name > on it (which felt a little bit too much like taking credit for other > people's work). My intention was never for that list of acknowledgements > to imply any sort of endorsement, though re-reading it now that might be a > little bit less explicit than I'd thought; for example, I describe the list > as people who "shaped the Working Group's discussion". > > For example, you joined us at the last in-person interim in London, and > appear on the bluesheets for several virtual interims and other IETF > meetings. Max appears on the bluesheets for almost every AIPREF session > since the Working Group started, including main IETF meetings (122 Bangkok > virtually, 123 Madrid, 124 Montreal, 126 Vienna), most if not all of the > virtual interims, the London design team meeting, and the in-person > interims in Zurich, Toronto, and London. That felt worth acknowledging. > > I can't submit a new version of the draft right now because the > Datatracker is locked, but I'm happy to make changes. I'll make it more > explicit that acknowledgement does not imply endorsement, or participation > in my individual draft (versus participation in the Working Group, whose > work my draft borrows wholesale). If you would like me to redact your > affiliation, or to remove your name entirely, I'm happy to do that, to; > just let me know what you'd prefer. > > If Max or anyone else I listed is feeling similarly, then: I apologize in > advance; just shoot me a note off-list and let me know what you'd prefer. > > -K > > On Tue, Nov 10, 2026 at 1:34 PM Alex Gibson <agibson= > 40newscorp.com@dmarc.ietf.org> wrote: > >> All, >> >> Per the email below, Kevin Kelley of Anthropic has posted an >> Internet-Draft, "Machine-Readable Training Preference Signal," to the IETF >> AI Preferences Working Group page. Its Acknowledgements section lists Max >> Gendler and me, with our News Corp affiliation, thanking us for >> "substantial contributions to shaping the Working Group's discussion" and >> for our "partnership." >> >> Neither of us contributed to this draft or worked in partnership with Mr. >> Kelley or Anthropic. News Corp does not support the proposal. >> >> Please remove our names from the draft. Please also do not use our names, >> or News Corp's, in any way that suggests we took part in or support it. >> >> -- Alex Gibson, News Corp >> >>> >>> From: Kevin Kelley <kelleyk-ietf@kelleyk.net> >>> Date: Sat, 31 Oct 2026 at 2:39 am >>> Subject: [ai-control] New draft: >>> draft-kelley-aipref-training-preference-signal-00 >>> To: <ai-control@ietf.org> >>> >>> >>> Howdy everyone, >>> >>> I've just submitted a new I-D, >>> draft-kelley-aipref-training-preference-signal: >>> >>> >>> https://datatracker.ietf.org/doc/draft-kelley-aipref-training-preference-signal/ >>> <https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-kelley-aipref-training-preference-signal/__;!!F0Stn7g!DsFbaCWfaZxD1CdexVxtBKS5hVsyjYUqWPHnUdSpSHGnd2ruvnz_eHCHBMhthuUwBMpLXdDa2HSSlF0$> >>> >>> It describes a way to express AI-training preferences in robots.txt >>> without adding any new grammar. A site operator writes an ordinary rule >>> group whose User-Agent line names a "policy token": >>> >>> User-Agent: __usage_ai-training >>> Disallow: / >>> Allow: /press-kit/ >>> >>> This reads as "a preference against AI training for everything except >>> /press-kit/, and a preference in favor of it there." Lookup uses the usual >>> RFC 9309 path matching and longest-match precedence. >>> >>> This is a preference about use, not a traffic-management signal; the two >>> are intentionally separate, and ordinary rules for a crawler’s own product >>> token still govern fetching (though the draft does ask crawlers not to >>> fetch what they can’t use). >>> >>> This differs from the attachment draft that we put aside in order to >>> make progress on the vocabulary. Like I've mentioned, I've spent a lot of >>> time investigating how deployed parsers would interact with a wide variety >>> of ways of expressing preferences in robots.txt files. >>> >>> The mechanism in my draft uses only existing syntax, so parsers and >>> crawlers that don't implement it but comply with RFC 9309 should gracefully >>> ignore preference expressions and still receive other signals. It also >>> avoids relying on the areas where deployed parsers behave differently (and >>> let me tell you, there are plenty!). >>> >>> And it builds on prior art: some crawler operators already publish >>> tokens that control how content is used rather than which crawler can fetch >>> it. The draft introduces the term “policy token” as a vendor-neutral name >>> for that pattern, and calls the existing vendor-specific ones “proprietary >>> policy tokens”. >>> >>> Wildcard groups never match the new, vendor-neutral policy token, so >>> existing User-Agent: * rules aren’t read as expressing a preference, >>> which was a deliberate design goal. As a result, a robots.txt file that >>> doesn’t use this mechanism, as is true of every one published today, >>> expresses no preference at all. Rules for crawler product tokens and >>> proprietary policy tokens keep whatever meaning they have today. >>> >>> I’m proposing this as input to the WG’s attachment work, and I'm >>> planning to open a PR against the attachment draft to get thoughts from all >>> of you. I'll try to explain my thinking a little bit more there. (Thank >>> you to everyone who's already weighed in off-list!) >>> >>> On the vocabulary front, the policy token takes its meaning directly >>> from the "AI Training" definition in draft-ietf-aipref-vocab-08. >>> >>> I'm also excited to share that Anthropic has already started >>> implementing support for this draft. This complements our existing crawler >>> controls rather than replacing them. >>> >>> It'll take us some time to add support across our services and work >>> through the inevitable complications and edge cases that I'm sure we'll >>> discover. Also, as standards in this area evolve, how we interpret and >>> apply these signals may change. I'm looking forward to what we'll learn >>> from having "running code" in the field and how that will inform the work >>> we still have to do together. >>> >>> I'm very pleased that, over the coming months, all of our hard work will >>> start to become something that website publishers can use. Putting a >>> working signal in people's hands feels like tangible evidence that all of >>> the effort and heart we've put in, and all of the long hours and long >>> flights, are paying off. >>> >>> We hope that others will join us and make this a single signal that is >>> respected across the Internet. >>> >>> I'm curious to hear everyone's reactions and happy to answer any >>> questions. >>> >>> On a personal note: I'm incredibly grateful to everyone who has >>> invested time and effort in the WG's work and engaged in good faith, from >>> every point of view. I hope this is just the beginning of what we can ship >>> together, and I look forward to seeing all of you in San Francisco. >>> >>> Cheers, >>> >>> -K >>> _______________________________________________ >>> ai-control mailing list -- ai-control@ietf.org >>> To unsubscribe send an email to ai-control-leave@ietf.org >>> >> >> >> >> >> This message and its attachments may contain legally privileged or >> confidential information. It is intended solely for the named addressee. If >> you are not the addressee indicated in this message (or responsible for >> delivery of the message to the addressee), you may not copy or deliver this >> message or its attachments to anyone. Rather, you should permanently delete >> this message and its attachments and kindly notify the sender by reply >> e-mail. Any content of this message and its attachments that does not >> relate to the official business of News Corporation or its subsidiaries >> must be taken not to have been sent or endorsed by any of them. No >> representation is made that this email or its attachments are without >> defect. >> _______________________________________________ >> ai-control mailing list -- ai-control@ietf.org >> To unsubscribe send an email to ai-control-leave@ietf.org >> >
- [ai-control] New draft: draft-kelley-aipref-train… Kevin Kelley
- [ai-control] Re: New draft: draft-kelley-aipref-t… Laurent Le Meur
- [ai-control] Re: New draft: draft-kelley-aipref-t… Nate Hake
- [ai-control] Re: New draft: draft-kelley-aipref-t… Alissa Cooper
- [ai-control] No preference, default "allowed", pr… JF Farny
- [ai-control] Re: New draft: draft-kelley-aipref-t… Mark Nottingham
- [ai-control] Which party does a robots.txt readin… JF Farny
- [ai-control] Re: New draft: draft-kelley-aipref-t… Alex Gibson
- [ai-control] Re: New draft: draft-kelley-aipref-t… Kevin Kelley
- [ai-control] Re: New draft: draft-kelley-aipref-t… Kevin Kelley
- [ai-control] Re: New draft: draft-kelley-aipref-t… Nate Hake
- [ai-control] Re: New draft: draft-kelley-aipref-t… Srecko Jovancevic
- [ai-control] Re: New draft: draft-kelley-aipref-t… angela.mills-wade
- [ai-control] Re: New draft: draft-kelley-aipref-t… Martin Thomson
- [ai-control] Re: New draft: draft-kelley-aipref-t… Alex Gibson
- [ai-control] Re: New draft: draft-kelley-aipref-t… Kevin Kelley
- [ai-control] Re: New draft: draft-kelley-aipref-t… Kevin Kelley