[ai-control] Re: [Question] Priority conflict between HTTP Header and Robots.txt in draft-ietf-aipref-attach

[Tyler Martin <[e-mail]>]

I agree with John. Precedence should follow signaling specificity, not attachment mechanism. A resource-level declaration (HTTP header or meta tag) is strictly more specific than a path-level robots.txt rule and should therefore override it.

This mirrors how developers already reason about cascading systems like CSS: broader scopes establish defaults; more specific scopes provide explicit overrides. That model is predictable, composable, and avoids accidental overreach from path-level declarations.

Encoding specificity ordering directly in the vocabulary would give site operators flexibility while preserving determinism for automated systems.

Regards,
Tyler


Tyler Martin
Founder, ©Copyrightish - AI Web Content Licensing
[e-mail]
https://copyright.sh


> On Mar 2, 2026, at 10:00 AM, John Mueller <johnmu=[e-mail]> wrote:
> 
> I worry a bit that this means that robots.txt (or another path-level signaling mechanism) automatically rules over resource-level controls. Since this is not a crawling-control directive (these already exist), I feel the spec should be specific and make a call about the order of precedence by signaling mechanism's specificity, not just by value. 
> 
> In particular, a robots.txt directive for "/foo" is less specific than a meta-tag or HTTP header provided on the resource "/foo", since the robots.txt directive would also apply to /foo/bar, /foo?bar=buzz and /foobarbuzz and the resource-level signal is explicitly only attached to that resource. Site operators have more flexibility if they can use both path-level and resource-level mechanisms. This feels like something that should be covered in the vocabulary, not in the attachment mechanism part. 
> 
> Cheers
> John
> 
> 
> On Fri, Feb 27, 2026 at 12:17 PM Martin Thomson <[e-mail] <mailto:[e-mail]>> wrote:
>> https://datatracker.ietf.org/doc/html/draft-ietf-aipref-vocab#section-5.1 covers this question.
>> 
>> In the absence of specific instructions to the contrary, the precedence order is n > y > ?.  That is, an 'n' in any place takes effect; a 'y' only takes effect if there are no 'n' preferences; and '?' (or no preference known) is only the outcome if no preference is expressed in any place.
>> 
>> On Fri, Feb 27, 2026, at 20:26, happypants wrote:
>> > Hi everyone,
>> > I am a new participant from Nanchang univeristy, and I am currently 
>> > reviewing the `draft-ietf-aipref-attach` to understand how to associate 
>> > AI usage preferences with content.
>> > I have a question regarding _priority_*_ _*_conflicts_.
>> > As defined in the draft:
>> >  • Section 2 defines the `Content-Usage` header field for HTTP 
>> > responses.
>> >  • Section 3 defines the `Content-Usage` rule for the Robots Exclusion 
>> > Protocol (robots.txt).
>> > What should be the expected behavior if these two mechanisms provide 
>> > _conflicting_*_ _*_instructions_ for the same resource?
>> > For example:
>> >  • The HTTP header says: `Content-Usage: train-ai=y` (Allow training)
>> >  • But the robots.txt says: `Content-Usage: train-ai=n` (Disallow 
>> > training)
>> > Which one should the automated system (crawler/AI) prioritize? Is there 
>> > a defined hierarchy, or should both be considered simultaneously?
>> > This is a practical scenario I am curious about, especially for 
>> > webmasters who might configure these separately.
>> > Thanks for any clarification!
>> > Best regards,
>> > Yudong Sheng
>> >
>> > happypants
>> > [e-mail] <mailto:[e-mail]>
>> >  
>> > <https://wx.mail.qq.com/home/index?t=readmail_businesscard_midpage&nocheck=true&name=happypants&icon=http%3A%2F%2Fthirdqq.qlogo.cn%2Fek_qqapp%2FAQHz6dqxVaO6JYUnqjw6FxFYfgLBwJ5x06N5q4T4NvpfXmpmnplWujm1by64gETo5y2jCcH0%2F0&mail=1195600347%40qq.com&code=To5No9HLC5CMpikw--yjC6Nfr_JzgfS1UPC4hrMlW5vhXQcywAKzi_elcDMeFBS77HO_N0sg6PBvws3kETB_DQ>
>> > -- 
>> > ai-control mailing list -- [e-mail] <mailto:[e-mail]>
>> > To unsubscribe send an email to [e-mail] <mailto:[e-mail]>
>> 
>> -- 
>> ai-control mailing list -- [e-mail] <mailto:[e-mail]>
>> To unsubscribe send an email to [e-mail] <mailto:[e-mail]>
> -- 
> ai-control mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]