Re: [Ietf-and-github] Mirja Kühlewind's No Objection on draft-ietf-git-using-github-05: (with COMMENT)

[Mirja Kuehlewind <[e-mail]>]

Hi Martin,

See inline.

> On 11. Mar 2020, at 07:50, Martin Thomson <[e-mail]> wrote:
> 
> Hi Mirja,
> 
> Thanks for taking the time to read through this.  I appreciate the input..
> 
> I've made some changes in https://github.com/ietf-gitwg/using-github/pull/45 but we might want to iterate on that some more.
> 
> On Tue, Mar 10, 2020, at 23:46, Mirja Kühlewind via Datatracker wrote:
>> I don't really understand the relationship between this document and
>> draft-ietf-git-github-wg-configuration. 
> 
> I think that document encompasses most of the stuff that I don't believe is appropriate for a BCP, so I think that the split is useful.  I think that others have addressed questions about the suitability of that document for publication.

I think having a split would be fine (also there is a bit of a question if draft-ietf-git-github-wg-configuration needs to be published as RFC), however, this doesn’t seem like a clear split because there is a lot of redundant requirements in this docent in section 2 (as well as some sentences in 3) and I don’t see a value in stating it twice in two different documents (especially as this one seems to be the one that states it normatively). 

> 
>> 1) Sec 3.2
>> "...or they might create repositories for individual documents on request."
>> I made a similar comment in my ballot for
>> draft-ietf-git-github-wg-configuration. I think creating repos for individuals
>> document is maybe not always the best option. I assume the intention here to
>> not out-rule any cases where this might be okay (e.g. if a document is expectd
>> to be adopted soon), however, I would rather prefer to see a recommendation to
>> usually not do that as there can be many individual drafts in the groups and it
>> can provide a wrong signal if repos are created for some and not others.
> 
> The intention is to not be specific.  What I have heard, from multiple Working Group chairs, is that they have allowed people to create repos for individual contributions in the organization and that they - and their contributors - have found it to be valuable.  Chairs who do this seem to be quite open about allowing anyone to request a repo.  That seems like a healthy balance.
> 
> I think that is good, because it removes any implied status involved in having a draft repo in the org.  I personally have fewer issues with moving repos than Warren does, so maybe I don't mind keeping things private until they are adopted.  Other do report challenges though.

There is a risk if chairs allow repos for some draft but not other, that those drafts are seen as more important. So I think some more discussion is needed in this document that if a wg decide to support individual drafts in their GitHub organisation, they are expected to give that opportunity to everybody asking.

However, I still see some higher risk here for confusing for people who are less familiar with the IETF and the respective naming convention. In the datatracker the document for each wg are clearly separated into a section for wg doc and other related docs. That is less obvious on GitHub, especially for people who only find things over GitHub and are not aware of e.g. the datatracker or other IETF processes.

I’m not saying we should forbid this. Chairs are free in how they think they can manage their groups best. But I also think we should not recommend it. Also we should probably give more guidelines how to make it more clear that documents have different status in GitHub besides the draft name. E.g. there is the repo description that could include a “warning”. Or we could recommend to have all wg doc repos as pinned repos but other not…



> 
>> 2) Sec 4.1.3:
>> "   Issues that have reached a resolution that has Working Group
>>   consensus MUST NOT be reopened unless new information is presented."
>> I do understand the intention here as it is important to make process, however,
>> inline with the rest of the document and the general idea that practises can
>> vary from group to group, I think this really should be a SHOULD NOT only.
> 
> This has been iterated on since after Barry's review.  Do the changes in https://github.com/ietf-gitwg/using-github/pull/43 work here?

Yes that’s much better. Thanks!

> 
>> 3) Sec 4.2:
>> "   Editors SHOULD make pull requests for all substantial changes rather
>>   than committing directly to the "master" branch of the repository."
>> I think this does not align well with how we work today (without GitHub). E.g.
>> if a group decides to use GitHub to better track changes and maybe easily
>> integrate editorial comments, but decide to have all discussion on the mailing
>> list (and not use github issues), I think it would be fully okay for the
>> editors just commit directly based on the consensus on the mailing list. So I
>> think the important part is that substantial comments should only be committed
>> with wg consensus and therefore it is a good practice to have pull requests
>> first to declare consensus based on the concrete proposed edits. However, if a
>> different way to declare consensus is used that's fine as well. In short, I
>> think the SHOULD is to string here for the general case.
> 
> I think that it greatly improves transparency, and I want to add that WG policies could insist upon this practice, but we don't need normative force in the two statements in this paragraph.

Yes, it should not be normative but I’m also not sure if that practice is the right thing for every document. Yes it greatly can improve transparency and I agree that is is a good practice but for some e.g. small documents it might be more overhead/“process burden” than really needed. So I think it would be good to  reword slightly and/or add some more emphasis that there might be cases where it might not be needed.

> 
>> 4) Sec 5.3 (mostly editorial): [...]
>> This text seems to apply more general to all content in section 5 and should
>> maybe be moved out of section 5.3. Similarly I'm not sure if section 5..3.1 and
>> 5.3.2 should actually be subsection of 5.3. The question of which issue to
>> track seem rather orthogonal to the question who resolves the issue (I agree
>> that this discussion does not make sense when no issue are tracked like in sec
>> 5.1 but it could make sense for the mode in section 5.2)
> 
> The organization here is tricky, but I do agree that this text is generic.  I've moved the piece you quoted up to the intro to Section 5.

Thanks! What’s about 5.3.1 and 5.3.1. I would recommend to create a new subsection there and not have these subsubsection within 5.3.
> 
> 
>> 5) Sec 5.3.1:
>> "The risk is that
>>   design changes might not always reflect the consensus of the Working
>>   Group."
>> I know that happens, even without GitHub, but should ideally not be the case.
>> However, I think the root cause is rather that changes were incorrectly not
>> identified by the editors as requiring consensus. Maybe this can be reworded
>> accordingly to not give the impression that it is okay to implement design
>> changes in a working group without consensus. Actually this is a broader
>> problem (independent of GitHub) because some editors, especially when they also
>> have been the authors of the individual draft, often implement design changes
>> first in a new version and then ask the working group for consensus. As I said
>> that's a different issue but it could be good to mention that GitHub and the
>> use of PRs can actually help to not use this practice but always each consensus
>> first.
> 
> I've added a piece about identifying the need for consensus, that is a good addition, but I don't see anything else actionable here.  I read the text as saying pretty much what you want it to.

Thanks! I would also propose to slightly rephrase it to make it more GitHub specific 
"The risk is that changes that potentially contain design changes might get merged into the working version of the document before consensus of the Working Group is confirmed. However, transparency of these changes is expected to still be higher compared to when documents are maintained by the editors without any detailed change log.”


> 
>> 6) Sec 5.3.1:
>> "... it is likely appropriate to move
>>   to a more tightly controlled process."
>> Maybe add something like "e.g. potentially during or after after working group
>> last call".
> 
> I don't want to prompt a particular outcome by making that sort of suggestion.  I appreciate that QUIC is weird, but it isn't THAT weird.  I've had several similar experiences in a relatively short period of time with HTTP/2, TLS, ACME, and a few others.

To be honest these groups are all “weird”, or rather to some extend similar. However, there are many groups that are much smaller scoped or at least work on less complicated protocols or only small protocol extensions. In all these cases the proposed tightly controlled process might be an over-kill (at least compared to how these group work currently without git). 

I don’t think this process is the best choice in general but rather one option. So I think it would be in general good to tone this part maybe a bit down. 

You can also replace “likely appropriate” with “may be useful” or something. My request is just to make this more a suggestion that a recommendation.

> 
>> 7) Sec 5.3.2:
>> "   Chairs might choose to manage the process of deciding which issues
>>   are substantive.  For instance, chairs might reserve the ability to
>>   use the "design" label to new issues (see Section 5.4.1) and to close
>>   issues marked as "design".  Chairs should always allow document
>>   editors to identify and address editorial issues as they see fit."
>> I guess you could also use normative language here: "MAY choose to" and "Chairs
>> SHOULD always allow"
> 
> I don't like normative clauses in a for example, but the latter seems good.

I suggested the MAY in the first sentence instead of might (which is not an example, no?)

> 
>> 8) Sec 5.3.2:
>> "   As documents mature further, explicit confirmation of technical
>>   decisions with the Working Group mailing list becomes more important."
>> Not sure I agree here. I know what you mean but explicit confirmation on the
>> mailing list is always important. Maybe there is a way to rephrase that (or
>> just remove that sentence...?)
> 
> This is really only emphasis, but I would prefer to keep it.

I’m concern that this sentence implies that technical design changes do not need to be confirmed in the early phase, which I think it just wrong. I think this is more than emphasis.

> 
>> 9) Sec 5.3.2:
>> "   Gaining Working Group consensus about the resolution of issues can be
>>   done in the abstract, with editors being permitted to capture the
>>   outcome of discussions as they see fit."
>> I'm actually not sure what you mean here. Can you clarify?
> 
> You can get consensus on specific language (usually in the form of a PR in this setting) or you can gain consensus on the outline of a design, without text.  In the past, we've been quite effective at the latter as well as the former.  Specific text is more concrete, but it can be better (particularly for documents in early stages) to just work out a rough outline first.

I believe you already revised this text based on other comments, right? (Lost track a bit). If so, maybe you can just point me add the next text…?

> 
>> 10) Sec 5.3.2:
>> "   WGLC SHOULD be proposed as pull requests, and MUST be discussed on
>>   the mailing list, and MUST have chairs explicitly confirm consensus.."
>> I agree with the "MUST be discussed on the mailing list" (as this is inline
>> with RFC2418 e.g. 3.2 but therefore doesn't necessarily need to be normative in
>> this doc) but find the other two SHOULD and MUST too strict. I don't think this
>> is aligned with the process we have today in groups and we should not use this
>> document to make the process more strict than it is. Especially I'm not sure
>> what "chairs MUST explicitly confirm consensus" really means and it seems to be
>> a requirement independent of GitHub and therefore eventually would even update
>> RFC2481.
> 
> How about:
> 
> Ideally, substantive changes to documents that have passed WGLC are
> proposed as pull requests, and MUST be discussed on the mailing list. Having
> chairs explicitly confirm consensus on changes ensures that previous consensus
> decisions are not overturned without cause.

Yes, thanks!

> 
>> 11) Sec 5.4.4:
>> It might be too late for this kind of input, however, as I review the document
>> I'll note it here anyway. In taps we also have a "discuss" label to mark issue
>> that has been discussed but need further discussion e.g. at an (interim)
>> meeting. For this, one could even create a milestone to indicate at which
>> meeting more discussion should take place (or when e.g. a document is
>> text-ready until when text should be provided). We/the editors also did this
>> for a while in taps.
> 
> We used labels like this in QUIC too.  Then we found that project boards were better for this. I think that someone suggested this during WGLC and we decided not to include it.

Okay.

> 
>> 12) Section 6 seems to encourage revision only in preparation for meetings. I'm
>> not sure if that is inline with our usual working practice. I mean yes in
>> reality we see the draft submission deadline as forcing function for updates
>> and there want to keep it but we also do encourage to rev documents more often
>> and work continuously, e.g. when a mayor change was implemented or a mayor
>> issue resolved. And yes this works probably better for smaller documents but
>> the section seems a bit contradicting to this practice. I mean the reason that
>> we see many updates just at the draft deadline is because editors assign time
>> to make updates to resolve issue to match the deadline. If editors make
>> continuous changes we should encourage to update more often. Maybe it could
>> rather say something like: "Revisions SHOULD be submitted as I-Ds when a
>> signification issue has been resolved. Editors MAY bundle multiple changes in
>> one revision if updates are done in timely close coherence and SHOULD update at
>> least two weeks before any meeting." However, some of this advise is actually
>> not GitHub specific and might again just touch our general guidelines and as
>> such update RFC2026 and RFC2418...
> 
> The intent here is to reiterate 2418 regarding session documents and to encourage routine publication of snapshots.

Actually it might be better to not use normative language here and rather point to 2418. However, right now the text does not seems to encourage routine publication of snapshots but rather it seems to encourage to “only” publish if there is a meeting.

> 
>> 13) sec 7:
>> "   Chairs MUST consider input from all discussion venues when assessing
>>   consensus including GitHub, mailing lists, interim meetings, and IETF
>>   meetings."
>> This seems like an update to RFC2481 again... as maybe also some other notes in
>> this section.
> 
> If you have concrete alternatives, I'm open to tweaking.  I think that as a restatement of existing principles, this is fine as is, but I'm sure that a different and better formulation is possible.

I think it should not use normative text here and rather point to rfc2481 instead. Maybe: 

“As specified in RFC2481, chairs must consider input from all discussion venues when assessing consensus. These include mailing lists, IETF meetings, interim meetings as well as issues and discussion on GitHub.”


Thanks!
Mirja



> 
>