WG Action: Formed Web Bot Auth (webbotauth)

[The IESG <[e-mail]>]

A new IETF WG has been formed in the Web and Internet Transport. For
additional information, please contact the Area Directors or the WG Chairs.

Web Bot Auth (webbotauth)
-----------------------------------------------------------------------
Current status: BOF WG

Chairs:
  Rifaat Shekh-Yusef <[e-mail]>
  David Schinazi <[e-mail]>

Assigned Area Director:
  Mike Bishop <[e-mail]>

Web and Internet Transport Directors:
  Gorry Fairhurst <[e-mail]>
  Mike Bishop <[e-mail]>

Mailing list:
  Address: [e-mail]
  To subscribe: https://www.ietf.org/mailman/listinfo/web-bot-auth
  Archive: https://mailarchive.ietf.org/arch/browse/web-bot-auth/

Group page: https://datatracker.ietf.org/group/webbotauth/

Charter: https://datatracker.ietf.org/doc/charter-ietf-webbotauth/

Automated clients (colloquially, ‘bots’) are increasingly used on the Web.
These clients may want to securely authenticate themselves as belonging to a
specific entity (a company or developer) or as being part of a specific
product (an AI bot, a search engine) for various reasons:

1. Origins wish to manage their resources and access control
2. Both bots and origins seek protection against impersonation and reputation
damage 3. Origins may wish to differentiate service levels between automated
and non-automated traffic

Current solutions (such as IP allowlisting, User-Agent strings, and shared
API keys) have significant limitations regarding security, scalability, and
manageability.

The Web Bot Authentication (webbotauth) Working Group will standardize
methods for cryptographically authenticating automated clients and providing
additional information about their operators to Web sites. Its products are
intended for use by sites that primarily serve human users.

# Scope

In-scope use cases include cryptographically authenticating access to Web
sites for: - Crawlers for search indices - Web archivers - Tools such as link
checkers and validators - Crawlers for AI training - AI agents retrieving or
interacting with content on behalf of end users

The following use cases are out of scope for this work:
- Authenticating access to content not intended for human consumption (e.g.,
HTTP APIs, agent-to-agent interfaces) - Authenticating the end user of a
participating client or agent - Authentication for application protocols
other than HTTP - Non-cryptographic authentication - Defining a vocabulary
for the intents of bots - Tracking or assigning reputation to particular bots
- Techniques for distinguishing non-participating bots from non-bot clients

There is significant industry work on "agents," where an automated client
makes requests on an end user's behalf. This effort will focus on
authentication of the agent; authentication of the end user is out-of-scope.

# Deliverables

The Working Group will deliver:
- Standards track document(s) describing technique(s) for authenticating
automated clients to Web sites intended for humans. - Standards track
document(s) describing a mechanism to convey more information about a
requesting bot using an existing widely-used identifier (such as a domain
name, hostname, or URL). - Best current practice and/or Informational
document(s) describing operational considerations such as lifecycle
management, key management, deployment considerations, etc. It will also
address impacts on the openness of the web.

The new authentication methods produced by this working group can add burden
to bot clients and web sites. The working group will consider this additional
burden, taking care to avoid architectural bottlenecks.

# Liaison

The Working Group is expected to liaise with the AIPREF, HTTPBIS, OAUTH, TLS,
and WIMSE Working Groups as appropriate on any relevant documents.

Milestones:

  Apr 2026 - Standards track specification(s) describing authentication
  technique(s) sent to the IESG

  Apr 2026 - Standards track specification(s) describing a means for
  conveying additional information about bots sent to the IESG

  Aug 2026 - Best Current Practice operational specification sent to the IESG