Re: [Ietf-and-github] Mirja Kühlewind's No Objection on draft-ietf-git-using-github-05: (with COMMENT)

[Martin Thomson <[e-mail]>]

Hi Mirja,

Thanks for taking the time to read through this.  I appreciate the input.

I've made some changes in https://github.com/ietf-gitwg/using-github/pull/45 but we might want to iterate on that some more.

On Tue, Mar 10, 2020, at 23:46, Mirja Kühlewind via Datatracker wrote:
> I don't really understand the relationship between this document and
> draft-ietf-git-github-wg-configuration. 

I think that document encompasses most of the stuff that I don't believe is appropriate for a BCP, so I think that the split is useful.  I think that others have addressed questions about the suitability of that document for publication.

> 1) Sec 3.2
> "...or they might create repositories for individual documents on request."
> I made a similar comment in my ballot for
> draft-ietf-git-github-wg-configuration. I think creating repos for individuals
> document is maybe not always the best option. I assume the intention here to
> not out-rule any cases where this might be okay (e.g. if a document is expectd
> to be adopted soon), however, I would rather prefer to see a recommendation to
> usually not do that as there can be many individual drafts in the groups and it
> can provide a wrong signal if repos are created for some and not others.

The intention is to not be specific.  What I have heard, from multiple Working Group chairs, is that they have allowed people to create repos for individual contributions in the organization and that they - and their contributors - have found it to be valuable.  Chairs who do this seem to be quite open about allowing anyone to request a repo.  That seems like a healthy balance.

I think that is good, because it removes any implied status involved in having a draft repo in the org.  I personally have fewer issues with moving repos than Warren does, so maybe I don't mind keeping things private until they are adopted.  Other do report challenges though.

> 2) Sec 4.1.3:
> "   Issues that have reached a resolution that has Working Group
>    consensus MUST NOT be reopened unless new information is presented."
> I do understand the intention here as it is important to make process, however,
> inline with the rest of the document and the general idea that practises can
> vary from group to group, I think this really should be a SHOULD NOT only.

This has been iterated on since after Barry's review.  Do the changes in https://github.com/ietf-gitwg/using-github/pull/43 work here?

> 3) Sec 4.2:
> "   Editors SHOULD make pull requests for all substantial changes rather
>    than committing directly to the "master" branch of the repository."
> I think this does not align well with how we work today (without GitHub). E.g.
> if a group decides to use GitHub to better track changes and maybe easily
> integrate editorial comments, but decide to have all discussion on the mailing
> list (and not use github issues), I think it would be fully okay for the
> editors just commit directly based on the consensus on the mailing list. So I
> think the important part is that substantial comments should only be committed
> with wg consensus and therefore it is a good practice to have pull requests
> first to declare consensus based on the concrete proposed edits. However, if a
> different way to declare consensus is used that's fine as well. In short, I
> think the SHOULD is to string here for the general case.

I think that it greatly improves transparency, and I want to add that WG policies could insist upon this practice, but we don't need normative force in the two statements in this paragraph.

> 4) Sec 5.3 (mostly editorial): [...]
> This text seems to apply more general to all content in section 5 and should
> maybe be moved out of section 5.3. Similarly I'm not sure if section 5.3.1 and
> 5.3.2 should actually be subsection of 5.3. The question of which issue to
> track seem rather orthogonal to the question who resolves the issue (I agree
> that this discussion does not make sense when no issue are tracked like in sec
> 5.1 but it could make sense for the mode in section 5.2)

The organization here is tricky, but I do agree that this text is generic.  I've moved the piece you quoted up to the intro to Section 5.


> 5) Sec 5.3.1:
> "The risk is that
>    design changes might not always reflect the consensus of the Working
>    Group."
> I know that happens, even without GitHub, but should ideally not be the case.
> However, I think the root cause is rather that changes were incorrectly not
> identified by the editors as requiring consensus. Maybe this can be reworded
> accordingly to not give the impression that it is okay to implement design
> changes in a working group without consensus. Actually this is a broader
> problem (independent of GitHub) because some editors, especially when they also
> have been the authors of the individual draft, often implement design changes
> first in a new version and then ask the working group for consensus. As I said
> that's a different issue but it could be good to mention that GitHub and the
> use of PRs can actually help to not use this practice but always each consensus
> first.

I've added a piece about identifying the need for consensus, that is a good addition, but I don't see anything else actionable here.  I read the text as saying pretty much what you want it to.

> 6) Sec 5.3.1:
> "... it is likely appropriate to move
>    to a more tightly controlled process."
> Maybe add something like "e.g. potentially during or after after working group
> last call".

I don't want to prompt a particular outcome by making that sort of suggestion.  I appreciate that QUIC is weird, but it isn't THAT weird.  I've had several similar experiences in a relatively short period of time with HTTP/2, TLS, ACME, and a few others.

> 7) Sec 5.3.2:
> "   Chairs might choose to manage the process of deciding which issues
>    are substantive.  For instance, chairs might reserve the ability to
>    use the "design" label to new issues (see Section 5.4.1) and to close
>    issues marked as "design".  Chairs should always allow document
>    editors to identify and address editorial issues as they see fit."
> I guess you could also use normative language here: "MAY choose to" and "Chairs
> SHOULD always allow"

I don't like normative clauses in a for example, but the latter seems good.

> 8) Sec 5.3.2:
> "   As documents mature further, explicit confirmation of technical
>    decisions with the Working Group mailing list becomes more important."
> Not sure I agree here. I know what you mean but explicit confirmation on the
> mailing list is always important. Maybe there is a way to rephrase that (or
> just remove that sentence...?)

This is really only emphasis, but I would prefer to keep it.

> 9) Sec 5.3.2:
> "   Gaining Working Group consensus about the resolution of issues can be
>    done in the abstract, with editors being permitted to capture the
>    outcome of discussions as they see fit."
> I'm actually not sure what you mean here. Can you clarify?

You can get consensus on specific language (usually in the form of a PR in this setting) or you can gain consensus on the outline of a design, without text.  In the past, we've been quite effective at the latter as well as the former.  Specific text is more concrete, but it can be better (particularly for documents in early stages) to just work out a rough outline first.

> 10) Sec 5.3.2:
> "   WGLC SHOULD be proposed as pull requests, and MUST be discussed on
>    the mailing list, and MUST have chairs explicitly confirm consensus."
> I agree with the "MUST be discussed on the mailing list" (as this is inline
> with RFC2418 e.g. 3.2 but therefore doesn't necessarily need to be normative in
> this doc) but find the other two SHOULD and MUST too strict. I don't think this
> is aligned with the process we have today in groups and we should not use this
> document to make the process more strict than it is. Especially I'm not sure
> what "chairs MUST explicitly confirm consensus" really means and it seems to be
> a requirement independent of GitHub and therefore eventually would even update
> RFC2481.

How about:

Ideally, substantive changes to documents that have passed WGLC are
proposed as pull requests, and MUST be discussed on the mailing list. Having
chairs explicitly confirm consensus on changes ensures that previous consensus
decisions are not overturned without cause.

> 11) Sec 5.4.4:
> It might be too late for this kind of input, however, as I review the document
> I'll note it here anyway. In taps we also have a "discuss" label to mark issue
> that has been discussed but need further discussion e.g. at an (interim)
> meeting. For this, one could even create a milestone to indicate at which
> meeting more discussion should take place (or when e.g. a document is
> text-ready until when text should be provided). We/the editors also did this
> for a while in taps.

We used labels like this in QUIC too.  Then we found that project boards were better for this. I think that someone suggested this during WGLC and we decided not to include it.

> 12) Section 6 seems to encourage revision only in preparation for meetings. I'm
> not sure if that is inline with our usual working practice. I mean yes in
> reality we see the draft submission deadline as forcing function for updates
> and there want to keep it but we also do encourage to rev documents more often
> and work continuously, e.g. when a mayor change was implemented or a mayor
> issue resolved. And yes this works probably better for smaller documents but
> the section seems a bit contradicting to this practice. I mean the reason that
> we see many updates just at the draft deadline is because editors assign time
> to make updates to resolve issue to match the deadline. If editors make
> continuous changes we should encourage to update more often. Maybe it could
> rather say something like: "Revisions SHOULD be submitted as I-Ds when a
> signification issue has been resolved. Editors MAY bundle multiple changes in
> one revision if updates are done in timely close coherence and SHOULD update at
> least two weeks before any meeting." However, some of this advise is actually
> not GitHub specific and might again just touch our general guidelines and as
> such update RFC2026 and RFC2418...

The intent here is to reiterate 2418 regarding session documents and to encourage routine publication of snapshots.

> 13) sec 7:
> "   Chairs MUST consider input from all discussion venues when assessing
>    consensus including GitHub, mailing lists, interim meetings, and IETF
>    meetings."
> This seems like an update to RFC2481 again... as maybe also some other notes in
> this section.

If you have concrete alternatives, I'm open to tweaking.  I think that as a restatement of existing principles, this is fine as is, but I'm sure that a different and better formulation is possible.