Return-Path: X-Original-To: ai-control@mail2.ietf.org Delivered-To: ai-control@mail2.ietf.org Received: from localhost (localhost [127.0.0.1]) by mail2.ietf.org (Postfix) with ESMTP id 0E4BAC1A05D1 for ; Mon, 2 Mar 2026 01:13:12 -0800 (PST) X-Virus-Scanned: amavisd-new at ietf.org X-Spam-Flag: NO X-Spam-Score: -0.727 X-Spam-Level: X-Spam-Status: No, score=-0.727 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_PASS=-0.001, URI_DOTCN_SPOOF=2.071] autolearn=no autolearn_force=no Authentication-Results: mail2.ietf.org (amavisd-new); dkim=pass (2048-bit key) header.d=copyright.sh Received: from mail2.ietf.org ([166.84.6.31]) by localhost (mail2.ietf.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a_R0pDkNSdWd for ; Mon, 2 Mar 2026 01:13:09 -0800 (PST) Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by mail2.ietf.org (Postfix) with ESMTPS id 755BBC1A05C8 for ; Mon, 2 Mar 2026 01:13:09 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=copyright.sh; s=key1; t=1772442780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TQWc1Mae57O15NlcV3iG1TgBMWYzY1XNOCYgvV+oi+4=; b=g3AU0Popjgn+2E3E2Ec/rVV9vyLwE9L9hFPpQP58i1rTAs6FYDsTZwqxZmZAbY8uTg93pe SR0MwpuHRQQkXbjMMn7bcaAkWwzp8SAczzUlf2KFH5UiVXoCxKlYJ89LnasussFU6Z17Q4 6X2uzILpzwGYCjl9oyBTk6UlXrBPDyC77IoDuTk700RhPF6RZzShCyyytHcIYTVnalXbzt gUDpXyIvlyLzwp19sjqGrZp4OnPjncfAsejReAFrZSJa2M0sdex3l+WpehKDoidpz8Tg+G IzplX65BJpSFo4Zy6lO/sQfUxbrvXDzGFITqfR1y23uJIuhUzgsHsIb/DXuNbQ== From: Tyler Martin Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_9648D9DF-40C0-45C6-AEF5-F3692FF92E5F" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Date: Mon, 2 Mar 2026 10:12:40 +0100 In-Reply-To: To: John Mueller References: <28040db0-6d16-4b4a-8f91-0586060a7fcf@betaapp.fastmail.com> X-Migadu-Flow: FLOW_OUT Message-ID-Hash: ORH726BUQUKYSZFOBSY3JEYRSRLGPB45 X-Message-ID-Hash: ORH726BUQUKYSZFOBSY3JEYRSRLGPB45 X-MailFrom: tyler@copyright.sh X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Martin Thomson , happypants <1195600347=40qq.com@dmarc.ietf.org>, "ai-control@ietf.org" X-Mailman-Version: 3.3.9rc6 Precedence: list Subject: =?utf-8?q?=5Bai-control=5D_Re=3A_=5BQuestion=5D_Priority_conflict_between_HT?= =?utf-8?q?TP_Header_and_Robots=2Etxt_in_draft-ietf-aipref-attach?= List-Id: AI Control Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --Apple-Mail=_9648D9DF-40C0-45C6-AEF5-F3692FF92E5F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I agree with John. Precedence should follow signaling specificity, not = attachment mechanism. A resource-level declaration (HTTP header or meta = tag) is strictly more specific than a path-level robots.txt rule and = should therefore override it. This mirrors how developers already reason about cascading systems like = CSS: broader scopes establish defaults; more specific scopes provide = explicit overrides. That model is predictable, composable, and avoids = accidental overreach from path-level declarations. Encoding specificity ordering directly in the vocabulary would give site = operators flexibility while preserving determinism for automated = systems. Regards, Tyler Tyler Martin Founder, =C2=A9Copyrightish - AI Web Content Licensing tyler@copyright.sh https://copyright.sh > On Mar 2, 2026, at 10:00=E2=80=AFAM, John Mueller = wrote: >=20 > I worry a bit that this means that robots.txt (or another path-level = signaling mechanism) automatically rules over resource-level controls. = Since this is not a crawling-control directive (these already exist), I = feel the spec should be specific and make a call about the order of = precedence by signaling mechanism's specificity, not just by value.=20 >=20 > In particular, a robots.txt directive for "/foo" is less specific than = a meta-tag or HTTP header provided on the resource "/foo", since the = robots.txt directive would also apply to /foo/bar, /foo?bar=3Dbuzz and = /foobarbuzz and the resource-level signal is explicitly only attached to = that resource. Site operators have more flexibility if they can use both = path-level and resource-level mechanisms. This feels like something that = should be covered in the vocabulary, not in the attachment mechanism = part.=20 >=20 > Cheers > John >=20 >=20 > On Fri, Feb 27, 2026 at 12:17=E2=80=AFPM Martin Thomson = > wrote: >> = https://datatracker.ietf.org/doc/html/draft-ietf-aipref-vocab#section-5.1 = covers this question. >>=20 >> In the absence of specific instructions to the contrary, the = precedence order is n > y > ?. That is, an 'n' in any place takes = effect; a 'y' only takes effect if there are no 'n' preferences; and '?' = (or no preference known) is only the outcome if no preference is = expressed in any place. >>=20 >> On Fri, Feb 27, 2026, at 20:26, happypants wrote: >> > Hi everyone, >> > I am a new participant from Nanchang univeristy, and I am currently=20= >> > reviewing the `draft-ietf-aipref-attach` to understand how to = associate=20 >> > AI usage preferences with content. >> > I have a question regarding _priority_*_ _*_conflicts_. >> > As defined in the draft: >> > =E2=80=A2 Section 2 defines the `Content-Usage` header field for = HTTP=20 >> > responses. >> > =E2=80=A2 Section 3 defines the `Content-Usage` rule for the = Robots Exclusion=20 >> > Protocol (robots.txt). >> > What should be the expected behavior if these two mechanisms = provide=20 >> > _conflicting_*_ _*_instructions_ for the same resource? >> > For example: >> > =E2=80=A2 The HTTP header says: `Content-Usage: train-ai=3Dy` = (Allow training) >> > =E2=80=A2 But the robots.txt says: `Content-Usage: train-ai=3Dn` = (Disallow=20 >> > training) >> > Which one should the automated system (crawler/AI) prioritize? Is = there=20 >> > a defined hierarchy, or should both be considered simultaneously? >> > This is a practical scenario I am curious about, especially for=20 >> > webmasters who might configure these separately. >> > Thanks for any clarification! >> > Best regards, >> > Yudong Sheng >> > >> > happypants >> > 1195600347@qq.com >> > =20 >> > = >> > --=20 >> > ai-control mailing list -- ai-control@ietf.org = >> > To unsubscribe send an email to ai-control-leave@ietf.org = >>=20 >> --=20 >> ai-control mailing list -- ai-control@ietf.org = >> To unsubscribe send an email to ai-control-leave@ietf.org = > --=20 > ai-control mailing list -- ai-control@ietf.org > To unsubscribe send an email to ai-control-leave@ietf.org --Apple-Mail=_9648D9DF-40C0-45C6-AEF5-F3692FF92E5F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
I agree = with John. Precedence should follow signaling specificity, not = attachment mechanism. A resource-level declaration (HTTP header or meta = tag) is strictly more specific than a path-level robots.txt rule and = should therefore override it.

This mirrors how = developers already reason about cascading systems like CSS: broader = scopes establish defaults; more specific scopes provide explicit = overrides. That model is predictable, composable, and avoids accidental = overreach from path-level = declarations.

Encoding specificity ordering = directly in the vocabulary would give site operators flexibility while = preserving determinism for automated = systems.

Regards,
Tyler


Tyler Martin
Founder, =C2=A9Copyrightish = - AI Web Content = Licensing
tyler@copyright.sh
https://copyright.sh


On Mar 2, 2026, at 10:00=E2=80=AFA= M, John Mueller <johnmu=3D40google.com@dmarc.ietf.org> = wrote:

I worry a bit that this means that robots.txt (or = another path-level signaling mechanism) automatically rules over = resource-level controls. Since this is not a crawling-control directive = (these already exist), I feel the spec should be specific and make a = call about the order of precedence by signaling mechanism's specificity, = not just by value. 

In particular, a = robots.txt directive for "/foo" is less specific than a meta-tag or HTTP = header provided on the resource "/foo", since the robots.txt directive = would also apply to /foo/bar, /foo?bar=3Dbuzz and /foobarbuzz and the = resource-level signal is explicitly only attached to that = resource. Site operators have more flexibility if they can use both = path-level and resource-level mechanisms. This feels like something that = should be covered in the vocabulary, not in the attachment mechanism = part. 

Cheers
John
<= br>

On Fri, Feb = 27, 2026 at 12:17=E2=80=AFPM Martin Thomson <mt@lowentropy.net> = wrote:
https://datatracker.ietf.org/doc/html/draft-ietf-aipref-= vocab#section-5.1 covers this question.

In the absence of specific instructions to the contrary, the precedence = order is n > y > ?.  That is, an 'n' in any place takes = effect; a 'y' only takes effect if there are no 'n' preferences; and '?' = (or no preference known) is only the outcome if no preference is = expressed in any place.

On Fri, Feb 27, 2026, at 20:26, happypants wrote:
> Hi everyone,
> I am a new participant from Nanchang univeristy, and I am currently =
> reviewing the `draft-ietf-aipref-attach` to understand how to = associate
> AI usage preferences with content.
> I have a question regarding _priority_*_ _*_conflicts_.
> As defined in the draft:
>  =E2=80=A2 Section 2 defines the `Content-Usage` header field = for HTTP
> responses.
>  =E2=80=A2 Section 3 defines the `Content-Usage` rule for the = Robots Exclusion
> Protocol (robots.txt).
> What should be the expected behavior if these two mechanisms = provide
> _conflicting_*_ _*_instructions_ for the same resource?
> For example:
>  =E2=80=A2 The HTTP header says: `Content-Usage: train-ai=3Dy` = (Allow training)
>  =E2=80=A2 But the robots.txt says: `Content-Usage: = train-ai=3Dn` (Disallow
> training)
> Which one should the automated system (crawler/AI) prioritize? Is = there
> a defined hierarchy, or should both be considered = simultaneously?
> This is a practical scenario I am curious about, especially for =
> webmasters who might configure these separately.
> Thanks for any clarification!
> Best regards,
> Yudong Sheng
>
> happypants
> 1195600347@qq.com
> 
> <https://wx.mail.qq.com/home/index?t=3Dreadmail_businessc= ard_midpage&nocheck=3Dtrue&name=3Dhappypants&icon=3Dhttp%3A%2F= %2Fthirdqq.qlogo.cn%2Fek_qqapp%2FAQHz6dqxVaO6JYUnqjw6FxFYfgLBwJ5x06N5q4T4N= vpfXmpmnplWujm1by64gETo5y2jCcH0%2F0&mail=3D1195600347%40qq.com&cod= e=3DTo5No9HLC5CMpikw--yjC6Nfr_JzgfS1UPC4hrMlW5vhXQcywAKzi_elcDMeFBS77HO_N0= sg6PBvws3kETB_DQ>
> --
> ai-control mailing list -- ai-control@ietf.org
> To unsubscribe send an email to ai-control-leave@ietf.org

--
ai-control mailing list -- ai-control@ietf.org
To unsubscribe send an email to ai-control-leave@ietf.org
--
ai-control mailing list -- ai-control@ietf.org
To unsubscribe = send an email to = ai-control-leave@ietf.org

= --Apple-Mail=_9648D9DF-40C0-45C6-AEF5-F3692FF92E5F--