Re: [Eligibility-discuss] Discussion of draft-ietf-elegy-rfc8989bis and IETF 115

[Donald Eastlake <[e-mail]>]

Hi,

I have read and reviewed the elergy draft. Below are my comments. My
apologies for not
making these comments earlier.


  -- elegy                                                            M.
Duke
  -- Internet-Draft                                                Google
LLC
  -- Obsoletes: 8989 (if approved)                          16 September
2022
  -- Updates: 8713 (if approved)

  -- Intended status: Best Current Practice

  -- Expires: 20 March 2023
  --
  --
  --                     Nominating Committee Eligibility
  --                      draft-ietf-elegy-rfc8989bis-00
  --
  -- Abstract
  --
  -- The IETF Nominating Committee (NomCom) appoints candidates to most
  -- IETF leadership committee.  RFC8713 provides criteria for membership
  -- on NomCom that attempt to ensure that NomCom volunteers are members
  -- of the loosely defined IETF community, by requiring in-person
  -- attendance in three of the past five in- person meetings.  In 2020
  -- and 2021, the IETF had six consecutive fully online plenary meetings
  -- that drove rapid advancement in remote meeting technologies and
  -- procedures, including an experiment that included remote attendance
  -- for NomCom eligibility.  This document updates RFC8713 by building a
  -- new set of eligibility criteria from first principles, with
  -- consideration for the increased salience of remote attendance.
  --
  -- Status of This Memo
  --
  -- ...
  --
  -- This Internet-Draft will expire on 20 March 2023.
  --
  -- Copyright Notice
  --
  -- Copyright (c) 2022 IETF Trust and the persons identified as the
  -- document authors.  All rights reserved.
  --
  -- ...
  --
  -- 1.  Introduction
  --
  -- [RFC8713] defines the process for selection of the Internet
  -- Architecture Board (IAB), Internet Engineering Steering Group (IESG),
  -- IETF Trust, and the IETF LLC Director.  These four committees form
  -- the senior leadership of the IETF.  A key actor in the process is the
  -- Nominating Committee (NomCom), which nominates a single candidate for
  -- each open position from the pool of volunteers, subject to

Remove "from the pool of volunteers". Seems confusing since this
document is about who qualifies to be in the "pool of volunteers for
the nomcom". Furthermore, the nomcom sometimes has to "draft" someone
who has not volunteered for office.


  -- confirmation by other bodies.
 --
  -- NomCom voting members are themselves volunteers that have met certain

Suggest deleting "themselves".


  -- eligibility requirements.  The actual NomCom is selected at random
  -- from the pool of eligible volunteers, with restrictions to ensure
  -- that no more than two volunteers with the same primary affiliation
  -- are chosen.

I don't see any need to mention the affiliation restriction in this
document which should just focus on nomcom volunteer pool eligibility.
Any additional restrictions on the random selection should stay in RFC
8713 and successors. Suggest replacing the last sentence with "The
actual NomCom is selected at random from the pool of eligible
volunteers.  Thus, it is important that members of the pool be IETF
participants likely to have knowledge of IETF processes and Tao."


  -- ...
  --
  -- Section 4.14 of [RFC8713] requires that volunteers must have attended
  -- three of the previous five in-person meetings.  In practice, this has
  -- meant that the volunteer picked up their registration badge.

Comment: While this has been true, I think it was because, with
hand-written blue sheets, it was impractical to get a significantly
better indication of attendance. The current hybrid meeting
technology with automatic blue sheets makes it much easier to enforce
a much more solid attendance requirement. See further comments below.


  --                                                               Current
  -- members of the Internet Society Board of Trustees, IETF Trust, LLC
  -- Board, IAB, and IESG are ineligible.

The set of bodies for which the nomcom nominates has changed and may
change in the future. Suggest minimizing the explicit list. So
"Current members of Internet Society Board of Trustees and the bodies
for which the nomcom nominates members are ineligible."


  -- [RFC8989] specified an experiment in the wake of six consecutive
  -- fully online meetings from 2020 to 2021, where the traditional
  -- interpretation of the requirement would have resulted in no eligible
  -- volunteers.  It extended the attendance requirement to define meeting
  -- attendance as including logging in to at least one session of a
  -- fully-online IETF meeting.

In my opinion, that is an absurdly lax attendance requirement for a
meeting. See further comments below.


  -- RFC8989 also created two other tracks to obtain eligibility: (1)
  -- serving as a working group chair or secretary in the past 3 years,
  -- and (2) author or editor of an IETF Stream RFC in the past five
  -- years, including internet-drafts in the RFC Editor queue.
  --
  -- This document discusses some of the first principles that inform the
  -- design of NomCom eligibility.  It makes recommendations on how the
  -- future process should work.  Its objective is to eventually replace
  -- Section 4.14 of RFC8713 with criteria loosely based on those in
  -- RFC8989.

Remove the word "eventually" -- isn't adoption of a version of this
document going to zap the criterion to what is specified herein?

Why would the eligibility criterion be loosely based on RFC 8989 rather
than loosely based on the first principles?


  -- 2.  Conventions and Definitions
  --
  -- ...
  --
  -- 3.  NomCom Principles
  --
  -- The NomCom is intended to be composed of randomly selected members of
  -- "the community."  For many years, in-person attendance was a
  -- reasonable proxy for the commitment associated with being a member.
  -- Two days of travel and an attendance fee is a relatively large
  -- expenditure of time and money.  Additionally, in-person attendance is
  -- thought to increase personal familiarity with candidates for
  -- leadership positions, although there is no mechanism to ensure any

"positions," -> "positions and with the spirit of the IETF,"


  -- interactions.  Finally, the NomCom interview process was largely
  -- conducted in-person at IETF meetings, so the ability to attend was a
  -- prerequisite to participate.
  --
  -- ...
  --
  -- Beyond the principle that the community should govern itself,
  -- selecting volunteers with a demonstrated commitment to the
  -- organization, while limiting the number from any organization, avoids
  -- the potential for mischief via nominations that disrupt IETF
  -- operations or work against the interests of the community as a whole.
  --
  -- However, attitudes to business travel evolve, and remote meeting
  -- technology continues to improve, to the extent that many longstanding
  -- community members choose to participate remotely.  The system has
  -- always excluded community members due to cost or personal reasons.

The above sentence is a little off. Could replace "community members"
with "some from attendance, and thus qualification for the nomcom".


  -- Further, the NomCom can now fully complete its business using online
  -- tools.
  --
  -- Counting remote attendance lowers the barriers to entry.

Add "but decreases the average knowledge of IETF processes, spirit,
and leadership by those eligible. Thus, a balance is required between
openness and qualifications."


  --                                                          As IETF is
  -- committed to having a no-fee remote option
  -- ([I-D.draft-ietf-shmoo-remote-fee]), the only required investment is
  -- to log on once per meeting at a specific time (sometimes a locally
  -- inconvenient hour).

The above sentence seems unnecessary. I don't see how it matters much,
for the purposes of this document, whether or not there is a no-fee
remote option nor whether the IETF is committed to it.


  --                      While this document does not formally impose a
  -- requirement for the NomCom to function entirely remotely, including
  -- remote-only attendees in the pool is likely to effectively require a
  -- remote component to NomCom operations.
  --
  -- Finally, it is historically difficult to recruit volunteers for
  -- NomCom, so overly restrictive criteria work against getting a deep
  -- talent pool.

In my opinion, the above sentence is just plain false. Volunteers to
serve on the nomcom are only hard to recruit if you have excessive
goals.  My opinion is that the nomcom worked fine for years and years
in its early times with a volunteer pool around 50 or 60. Even with
that a volunteer has only about an 18% chance of being chosen. I can
see someone wanting >100 in the pool. But I don't see any need for
>200.

The idea that the world is divided into "the IETF community" and "not
the IETF community" by a line seems wrong to me. There is really a
spectrum from people with deep, full-time, long-term participation, to
people who have just read a few messages on one IETF mailing list or
stopped by one meeting of one WG.

It seems to me that fundamental principles for nomcom pool eligibility
would be things like

 + A pool generally biased towards those more familiar with the
   spirit, processes, and leaders of the IETF.

 + A large enough pool membership that each pool member has < 10%
   chance of being chosen so they do not feel entitled to a vote.

 + A sufficiently dynamic pool, implemented through a sufficiently
   short time horizon, that you get a reasonable number of new pool
   members each year.


  -- 4.  Criteria
  --
  -- The following paths to qualification replace Section 4.14 of
  -- [RFC8713].  Any one of the paths is sufficient, unless the person is
  -- otherwise disqualified under Section 4.15 of [RFC8713].
  --
  -- Path 1: The person has registered for and attended 3 out of the last
  -- 5 IETF meetings, either in-person or online.  In-person attendance is
  -- as determined by the record keeping of the Secretariat.  Online
  -- attendance is based on being a registered person who logged in for at
  -- least one session of an IETF meeting.

Someone who logs in for only one session for a meeting seems like the
sort of ultra-narrow participant that we DO NOT want on the nomcom.
Furthermore, a reference to the "last 5 IETF meetings" is affected by
the number of meetings per year.  Better to say something like "3 IETF
meetings during the last 2 years". Remote attendance is OK and was
obviously necessary in the era of COVID but this "one session"
criterion is absurdly lax.

With automated blue sheets and hybrid meetings, even though in-person
attendance is more securely determined than on-line, I'm not sure how
much sense it makes to distinguish them. How about something more like
"The person has attended at least 3 IETF meetings over the previous 2
years. Attendance is defined as their attendance being recorded
(either on-line or in-person) for at least two sessions on each of at
least 3 days of the meeting."

(Although the criterion has been totally revised due to the era of
COVID, you might be interested in what the attendance requirements of
another standards organization were, in particular IEEE 802.11 WG
which has 6 weeklong meetings a year. Before COVID, as I recall, to
count as having attended for the week you have to be present for 75%
of the time slots (some slots were plenary while other time slots were
broken out by Task Group). And to count as having attended during a
time slot, you had to be in attendance for 90% of that time slot.
After attending for a couple of these weeks within a one-year interval
(it's actually a bit more complex than that), you qualified as a voter
and decisions were made by in-person voting.)


  -- Path 2: The person has been a Working Group Chair or Secretary within
  -- the 3 years prior to the day the call for NomCom volunteers is sent
  -- to the community.

I understand that very few people qualified under this path that
didn't qualify under Path 1.  Especially if that's true, then I see no
reason for the path -- there is plenty of representation of the WG
Chair viewpoint among others qualifying under Path 1. Each path adds
complexity and increases the attack surface. It also doesn't look that
good where the ADs/IESG select WG Chairs and the WG Chairs, even if
they haven't qualified in any other way, are eligible to be involved
in the selection of the ADs/IESG.

I didn't realize that WG Secretaries were included. This seems pretty
flakey to me. Some WG Secretaries do a fair amount of work, and some do
very little. They can just be appointed/removed by the Chairs. I have
a lot more confidence in the ADs/IESG than I do in, say, the worst of
the WG Chairs. Can a WG Chair appoint a new Secretary of their WG each
day for a month? I see no reason why not. And as far as I know,
neither the WG nor the AD is directly informed when a Secretary is
appointed.

Why should someone new to the IETF just appointed a Chair (presumably
paired with an experienced co-Chair) or Secretary and perhaps having
only served for a few weeks or even days be nomcom eligible if they
don't qualify in any other way?

I would just drop this path. (If it were to be kept, I would change it
to require time in office. So "... Chair [or Secretary] for at least a
year within the 2 years prior ...".)


  -- Path 3: The person has been a listed author or editor (on the front
  -- page) of at least two IETF Stream RFCs within the last 5 years prior
  -- to the day the call for NomCom volunteers is sent to the community.
  -- An Internet-Draft that has been approved by the IESG and is in the
  -- RFC Editor queue counts the same as a published RFC, with the
  -- relevant date being the date the draft was added to the RFC Editor
  -- queue.  For avoidance of doubt, the 5-year timer extends back to the
  -- date 5 years before the date when the call for NomCom volunteers is
  -- sent to the community.

I certainly think people who contribute to RFCs should get
recognition.  However, I think the nomcom eligibility pool should be
chosen primarily with the aim of getting the best nomcom, not
primarily as a reward for people who have contributed. You do
learn/see some things in putting a document through but mostly if you
are the primary author.

And 5 years is too long.  I think 2 or 3 years would be long
enough. Consider that documents can get stuck in misref for
years. Here, for example, is a document that has been stuck in misref
for almost 5 years:
https://datatracker.ietf.org/doc/search?name=trill-ecn&activedrafts=on
This sort of thing could stretch eligibility out for quite a while but
only in a few isolated cases.

Say someone is the world's leading expert of the WYX protocol. They
review a lot of drafts and provide excellent comments and even suggest
some text but someone else is always the primary author of drafts they
contribute to. And they are happy with that because they care about the
technicalities of the protocol and don't want to futz around with
documents. They only subscribe to the WYX email list and never attend
a meeting of anything but the WYX WG.  They get listed on the first
page of a number of RFCs. Why is it a good idea for them to be nomcom
eligible? And the current zeitgeist is the if you are eligible,
people are going to push, Push, PUSH for you to volunteer because they
seem to think a bigger and bigger and bigger volunteer pool is always
better even if the result is more people not that enthusiastic about
putting in the work required and more people declining if they are
randomly selected, etc.


So, I don't want to just be critical without providing an
alternative. I guess if I was drafting a new criterion to replace
these multiple paths, it would be something along the following line:

   The person has attended at least 3 IETF meetings over the previous
   2 years. Attendance is defined as recording your attendance (either
   on-line or in-person) at at least two sessions on each of at least
   3 days of the meeting. Being a front-page author of an IETF Stream
   RFC (or approved draft in the RFC Editor's queue) in the previous
   three years may be substituted for up to two of these attendance
   instances.

No doubt the above wording needs some tuning even if people like it.


  -- 5.  Available Data
  --
  -- TODO: This document should contain data about how the proposed
  -- criteria would have affected eligibility for NomComs in the recent
  -- past.
  --
  -- 6.  Security Considerations
  --
  -- The threat model associated with NomCom eligibility is that an
  -- organization or group of organizations would attempt to obtain a
  -- majority of NomCom positions, in order to select an IETF leadership
  -- in support of an agenda that might be self-serving and against the
  -- interests of the community as a whole.
  --
  -- Note that [RFC8713] lets the Chair decide the NomCom voting
  -- requirement, so a simple majority may be inadequate.  However, 7 of
  -- 10 forms a quorum, so at worst seven NomCom members working together
  -- can almost certainly impose their will.

I don't think the exact vote is particularly important.


  -- Whatever the merits of admitting remote attendees, it reduces the
  -- minimum cost of creating a NomCom-eligible volunteer from three
  -- flights and ~5 days of travel over the course of a year, to zero
  -- financial cost and the time required to log in three times over the
  -- course of a year.  Some organizations might not be deterred in either
  -- case, while others might now find such an attack to be feasible.

While it is not a huge change in the effort required, replacing the
absurdly lax attendance requirement with a stricter requirement, as I
have suggested above, does help a little.


  -- 6.1.  A Surge of Volunteers
  --
  -- A large number of "legitimate" volunteers makes it quite difficult to
  -- control 6 of 10 NomCom slots.  Setting aside limitations on the
  -- number of selections from any organization, basic probability shows
  -- that to have even a 50% chance of controlling 6 or more NomCom
  -- positions, an attacker needs somewhat roughly 60% of the volunteer
  -- pool.  For example, if there are 300 "legitimate" volunteers, an
  -- attacker must produce 365 volunteers to exceed a 50% chance of NomCom
  -- capture (see Appendix A).
  --
  -- A sudden surge in the number of volunteers, particularly of people
  -- that no one recognizes as a part of the community is an early-warning
  -- system for leadership to further investigate.

Sure, but who is supposed to notice this? And how does the leadership
intervene if they think there is an actual problem?


  -- While loosening eligibility criteria lowers the cost to an attacker
  -- of producing eligible volunteers, it also increases the number of
  -- "legitimate" volunteers that increases the difficulty and
  -- detectability of an attack.

The above seems correct except for the last bit. How does more
legitimate volunteers make an attack easier to detect? I would think
it provides a bigger dataset for the attack to hide in so I would
think it decreases the detectability of an attack.


  -- 6.2.  The Two-Per-Organization Limit
  --
  -- The two-per-organization limit in [RFC8713] complicates such an
  -- attack.  To circumvent it, an organization must either (1) coordinate
  -- with at least two like-minded organizations to produce a NomCom
  -- majority, (2) incentivize members of other organizations (possibly
  -- through a funding agreement) to support its agenda, or (3) propose
  -- candidates with false affiliations.

Nope. You are assuming some sort of "good faith" attackers.

Here is the sort of attack I would envisage: assume that Alphaland (a
fictitious country listed in RFC 3797) has a highly patriotic populace
and there is a large Technical University of Alphaland. They just go
to the campus and urge these patriotic students (and professors or
whatever), who all want to help Alphaland companies to have a higher
profile and more control in the IETF, to register for and attend WG
slots at IETF meetings. The at-most two with the same sponsor
provision has no effect: Each student or whoever just says they are a
one person consulting company and maybe the Technical University of
Alphaland has some work study courses under which the student actually
gets a consulting contract and gets paid a little. Once you remote /
virtualize things, it is a whole different ball game, and you have to
be very careful and/or have effective circuit breaker mechanisms
since, sooner or later, there will be abuse.


  -- While the IETF does not routinely confirm the affiliation of
  -- volunteers, as part of an investigation it could eliminate volunteers
  -- who have misrepresented said affiliation.  Publishing the list of
  -- volunteers and affiliations also gives the community an opportunity
  -- to review the truth of such claims.
  --
  -- Assuming that 300 legitimate volunteers are all from different
  -- organizations, three conspiring organizations would need 771
  -- volunteers (257 per organization) for a 50% chance of NomCom capture
  -- (see Appendix A).

As above, against a serious adversary, the 2 persons per sponsor limit
has no effect. I think this Section 6.2 should be dropped.


  -- 6.3.  One Year of Participation
  --
  -- Attendance at 3 meetings requires at least 1 year.  Given the volume
  -- of volunteers necessary to capture the process, an attack requires a
  -- surge in attendees over the course of a year.  IETF leadership SHOULD
  -- analyze unexplained surges in attendance to look for signs of
  -- manipulating the eligibility requirements (e.g. logging in to a
  -- single session and then immediately logging out).  In the event of
  -- malfeasance, the leadership would then have months to adjust policy
  -- in response before the NomCom cycle begins.

I think you want to say "abuse of process" or "manipulation" rather
than "malfeasance".  "Malfeasance" typically implies misconduct or
corruption by a public official.


I think a section is missing from the Security Considerations about
the greater difficulty of positive identification in a virtual
environment.  Something like the following:

   6.X.  Security of Identities

   Personal recognition in an in-person environment has always been
   acknowledged as the most security form of identification.  Remote
   access / virtualization makes secure identification of persons more
   difficult. It will be only a few years before real-time deep fake
   video software will be widely available on home computers.  There
   has already been one case of attempted IETF Working Group consensus
   manipulation through sock puppets. Nevertheless, it is felt that
   alertness to this issue and prompt investigation and, if warranted,
   action, will be a sufficient defense.


  -- 7.  IANA Considerations
  --
  -- This document has no IANA actions.
  --
  -- 8.  References
  --
  -- ...
  --
  -- Appendix A.  NomCom Capture Calculations
  --
  -- Section 6 offers some mathematical results for the probability of
  -- NomCom capture.  This appendix shows the work.
  --
  -- Let (a ch b) mean the number of combinations of b items chosen from a
  -- population of a items, or
  --
  -- (a ch b) = fact(a) / (fact(a-b) * fact(b))
  --
  -- A.1.  No per-organization limit
  --
  -- The first computation assumes there is no limit of two per
  -- organization, or equivalently, no organization produces more than two
  -- volunteers.
  --
  -- Let L be the number of "legitimate" volunteers (i.e. those not allied
  -- with an attacker" and A be the number of attacking volunteers.  Then
  -- there are ((L+A) ch 10) ways to select a NomCom.  The number of
  -- outcomes where attackers capture the NomCom is
  --
  -- Sum(i=6..10)[(A ch i) * (L ch (10-i)]
  --
  -- and the probability of capture is therefore
  --
  -- Sum(i=6..10)[(A ch i) * (L ch (10-i)] / ((L+A) ch 10).
  --
  -- For L = 300, this probability crosses 50% at A = 365.

I'm OK with A.1 but, as above, the two per sponsor limit has no effect
against a serious adversary so I think that Appendix A.2 should be
dropped and so the A.1 heading line can also be dropped leaving the
initial Appendix A material and the content of A.1.


  --A.2.  Two per Organization
  --
  -- Assume that the population of L is drawn from L different
  --
  -- ...
  --
  --Author's Address
  --
  -- Martin Duke
  -- Google LLC
  -- Email: [e-mail]


Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   [phone] (cell)
 2386 Panoramic Circle, Apopka, FL 32703 USA
 [e-mail]


On Tue, Oct 4, 2022 at 11:16 AM Barry Leiba <[e-mail]> wrote:

> For those who didn't attend the recent interim call: It was a fairly
> short call and an uneventful one, showing that we seem almost ready.
> We hope to make the IETF 115 session a wrap-up, and do a last call in
> the working group right after, so let's all work on preparing for
> that.
>
> First, as I said in the interim call and need to repeat here:
> As we already had community consensus on the three-path system that
> was in the experiment (which is also what's in the adopted draft), and
> as we're chartered to start our work from that, the chairs will be
> looking for consensus to change it, and assuming consensus on what's
> already there.  To be clear, all the details are up for discussion and
> change (though, as our charter says, the only changes in scope are the
> criteria for eligibility to volunteer for NomCom, which will replace
> Section 4.14 in the rules).  The point is simply that any details we
> do not have consensus to change will remain as they stand in the
> draft.
>
> Second, the agenda for the 115 session will be simple: Discuss and
> resolve any open issues on the draft and make sure we're ready to
> last-call it.  That means that we need to raise issues NOW, *before*
> the 115 meeting.  We will, of course, discuss anything within scope
> that comes up, but *please* do not wait for the meeting to read the
> document, and please raise any issues you have on this mailing list as
> soon as possible so we get the initial discussion going and so that we
> have a fully productive session at 115.
>
> All that said, we've got no issues in the queue right now.  On the
> interim call, Donald Eastlake said he has something he wants to
> propose and will write that up and post it here, so we await that (and
> this can serve as a reminder, Donald, to please do that right away).
> Anyone else: please keep your friendly and happy chairs both happy and
> friendly by posting soon and engaging in the discussion.
>
> Barry
>
> --
> Eligibility-discuss mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/eligibility-discuss
>