Feedback-ID: ie6694242:Fastmail
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\))
Subject: Re: Revised IETF LLM Tools
From: Mark Nottingham <mnot@mnot.net>
In-Reply-To: 
 <CAHBU6isyLbUqm82F5Nfi=5F6yYcrfztpZHSiXNrcnXO-36-6dw@mail.gmail.com>
Date: Tue, 14 Jul 2026 17:32:16 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <68BA0087-B66A-4E52-BC39-1F4FA742FAF3@mnot.net>
References: <7C5F050B-BE4C-4F11-81D5-6E044CE3FFC0@mnot.net>
 <946D14A0-2CCD-45C9-998B-C88F04136C36@mnot.net>
 <CAKoiRuahS=KiRVw49v4e31R-kZGKCCZPdBstFVpRQREM+exJaA@mail.gmail.com>
 <CAHBU6isyLbUqm82F5Nfi=5F6yYcrfztpZHSiXNrcnXO-36-6dw@mail.gmail.com>
To: Tim Bray <tbray@textuality.com>
Message-ID-Hash: RLECVE6WNB43UWQOF7RJPFLOZLECVREQ
CC: Rohan Mahy <rohan.mahy@gmail.com>, WG Chairs <wgchairs@ietf.org>
Precedence: list
Archived-At: 
 <https://mailarchive.ietf.org/arch/msg/wgchairs/9qiCjOQiXkrnmqunGcR_HO3zuaQ>

Hi Tim,

Like any tool, it absolutely can be misused. Deferring judgment to it is =
not what it's designed for, and the accompanying skills steer pretty =
hard away from that:
  https://github.com/mnot/ietf-skill
Of course, they can always be improved.

I started working on this because I observed a large and growing amount =
of LLM use in IETF participation, and think we need to have guiderails =
for its use. Part of that is carrot (ietf-llm and ietf-skill) and part =
of it is stick -- policies which the community needs to have a =
discussion about.

I'm talking about this more in RASPRG next week.

Cheers,


> On 14 Jul 2026, at 5:20=E2=80=AFpm, Tim Bray <tbray@textuality.com> =
wrote:
>=20
> Anyone but me think this whole idea is deeply misguided?  In most WGs =
on most occasions, consensus is obvious, and when it isn=E2=80=99t is =
when you absolutely don=E2=80=99t want to defer judgment to an LLM.
>=20
> But maybe I=E2=80=99m in the rough here. -Tim
>=20
> On Jul 14, 2026 at 8:07:46=E2=80=AFAM, Rohan Mahy =
<rohan.mahy@gmail.com> wrote:
>> Mark,
>> You email archive tool strips quoted lines, so how do you deal with =
inline comments and +1 on specific quoted paragraphs?
>>=20
>> thanks,
>> -rohan
>>=20
>> On Thu, 28 May 2026, 00:01 Mark Nottingham, =
<mnot=3D40mnot.net@dmarc.ietf.org> wrote:
>> Hey all,
>>=20
>> I've revised and renamed the tools below; it now includes a local MCP =
server. In my testing, it's pretty effective; you can chat away with a =
working group's corpus pretty to pretty good effect from eg Claude, =
OpenCode, etc.
>>=20
>>   https://pypi.org/project/ietf-llm/
>>=20
>> Still under development, but suggestions / issues welcome.
>>=20
>> Cheers,
>>=20
>>=20
>> > On 19 Mar 2026, at 11:07=E2=80=AFam, Mark Nottingham =
<mnot@mnot.net> wrote:
>> >=20
>> > Just thought I'd convey something I found useful for understanding =
the state of a WG's discussions.
>> >=20
>> > Google's NotebookLM:
>> >  https://notebooklm.google.com/
>> > allows you to ground a LLM in a specific set of documents.=20
>> >=20
>> > I've put all of the materials I have for the AIPREF Working Group =
into a notebook, including:
>> >=20
>> > - the Charter
>> > - meeting agendas
>> > - meeting minutes
>> > - drafts (including all revisions)
>> > - mailing list archives
>> >=20
>> > The result is an LLM I can chat with about the history and =
arguments made in the WG. Although (of course) I can't trust it =
completely, I'm finding it's a very good way to interact with the =
'memory' of a WG. In particular it can serve as 'another set of eyes' =
that doesn't have the biases that I may have.
>> >=20
>> > For the mailing list archive, I found that NotebookLM doesn't like =
just taking an mbox file, so I* wrote a script to pre-process an archive =
by stripping attachments, simplifying headers, and removing quoted =
material. See:
>> > https://gist.github.com/mnot/7c0c87f3bf3cc5bbcc97acb57eb1e332
>> >=20
>> > The main issue I have now is keeping it up-to-date.
>> >=20
>> > Hope this helps,
>> >=20
>> >=20
>> >=20
>> > * Gemini
>> >=20
>> > --
>> > Mark Nottingham   https://www.mnot.net/
>> >=20
>>=20
>> --
>> Mark Nottingham   https://www.mnot.net/
>>=20

--
Mark Nottingham   https://mnot.net/