[Errata-design] Raw notes from 22-January-2015 call

["Heather Flanagan (RFC Series Editor)" <[e-mail]>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Attending:
Joel, Barry, Pete, Stephen, Ted, Robert, Sandy, Alice, Nevil, Mark, Heather

Agenda

0. Administrivia
   - OK with most of the conversation via the list? - Yes
   - Team scheduling - expect this to go on for a few months, noting
that change may take more time since resources are limited
   - Participants from the IESG with March change over - no change in
participants (yay!)

1. Criteria for a successful errata system
    - "a measure of a successful errata system will be one
       visually and through metadata differentiate between
       levels of usefulness for an implementer/reader."
    - others?

Ted: Would add “consensus-ness” in addition to usefulness.  Some
suggestions are extensions to the document that do not reflect IETF
consensus.  On the other hand, there are typos, and those ought to be
fixed because EFL readers might not be able to recognize the typo; it
doesn’t impact the consensus.

Stephen: We should think about whether flagging certain thinks is
actually useful.  The errata system shouldn’t be a pain in the ass to use.

Robert: In place re-rendering of correction is a wet-ware piece.  

Stephen: if it really is important, someone will send email to the
IESG.  People can mod up what’s important.

Pete: the editing task for typos is in the RFC Editor; the wetware part
can be on their side of the wall.  The IESG shouldn’t have to be involved.

Stephen: a level of anti-spam in there is most helpful; if the RFC Editor

Sandy: the RPC could look at editorial errata, and could pass them on if
they’re not ok with making a judgement call.  There is a concern,
however, about RPC resources.  And how far back would this apply?

Ted: What if the way an erratum was submitted is that someone went in
and made an edit to a special version of the document to help the RFC
Editor then figure out what to apply.  

Pete: if what you’re doing is clicking and accepting an errata, and then
it goes to a display form, that’s in the noise on how we present it.  Is
the act of “yes this is editorial or no it should be thrown over the
wall” a reasonable amount of work for the RFC editor.  The criteria is
to differentiate levels of usefulness.  If all editorial errata of this
sort that the RPC approves go into what currently looks like the errata
system, and everything else goes into a community voting system, that
would be good.  The way that the errata of completely editorial sort
that we’re talking about can stay in the current system (no tooling
change) but we’ll differentiate the technical stuff.

Sandy: where is the priority?  To show the notification re: editorial
errata, or to show the technical?

Pete: the editorial stuff you will pull off the top and put in the
existing system, and everything else goes over the wall.  If it is a
serious enough problem, then it may need to go back over the wall to get
highlighted or displayed.  Things that need to be visible immediately
can go back to the RPC.  The whole businesses of all the other mush is
taken care of the wiki based discussion, and they will work out the issues.

Robert: we are conflating a few questions.  One, looking at what the
effort was and what the task was on deciding on whether to deal with the
errata as a typo or a technical errata.  Two, how do we present the
errata to people that need to consume it.  Pete pulled this apart, which
is good, but let’s keep them separated.

Stephen: should any of these errata be handled in the same way as legal
item?

Robert: Yes, as part of the presentation.  They could be patches, and
the patches get signed.  Coming back to Ted’s question to Sandy: If you
are evaluating a piece of errata to see if it’s technical or editorial,
is it easier to do what you do now with a before/after description, or
would it be easier if you were presented with an edited version of the
document and look at a diff?  If you had a patch, would you be
comfortable passing that patch through to render an edited version of
the document, or would you need to do your own tweaks to the edit?

Sandy: for the source form, if no one cares about the editorial errata,
no need to change the system we’re using.  The only place where it might
be beneficial to have the source form is when the submitters are
submitting bad text anyway (they are submitting errata against text that
doesn’t actually exist)

Joel: risk in providing long-form edits in source file.  People may
consider that an opportunity to make lots of edits.

Ted: The current errata system is not useful.  It may be useful for the
RPC, but it’s not useful for the community because it’s so difficult to
use it to find errata.

Sandy: understand the accessibility issue, and hope that will be
addressed in the new format era, that something will exist on the
outputs that errata exist.  Is there consensus that editorial errata
aren’t useful?  

Pete: they are important enough for some purposes, but not important
enough for the IETF to review.

Ted: agree that we should have a modding system for technical errata.
The debate here is about the editorial errata and their level of
importance.  Typos that change the meaning of the text that change the
meaning to an EFL speaker are important.

Stephen: Disagree that they are important.

Sandy: if we are trying to make a distinction between important (to EFL)
editorial errata versus not, then we have a challenge.

Barry: can we ask non native speakers to offer their own opinions as to
whether this is actual a problem?

Mark: do we have any data on the click-through rate for errata?

Joel: I check the errata when I’m actually implementing the specification.

Robert: Let’s ask AMS to run outsets awstats

Mark: Do we have any different processes we want to have for errata
filed against documents with still-active working groups?  It would be
helpful to immediately rope in the active WG.  But the small changes
against orphaned RFCs will just get lost.

Sandy: people also use the errata when -bis documents are written.  When
a -bis document comes in, the RPC checks to see if the errata have been
folded in.

Mark: would it be useful for WGs to sign up to take errata-level
ownership for orphaned RFCs?  (useful idea in principle, may not be
broadly applicable in practice; existing “orphaned” RFCs have assigned
areas)

Alice: people who submit editorial errata are often EFL speakers, and
there is some value in capturing editorial errata in that they’ll be
reported over and over again.  Even if you build a system that says
“don’t submit typos” they will continue to be submitted.  So the
question is how you want to display.  We don’t get many redundant support

Ted: it would be nice if they didn’t have to struggle for that.  If this
is happening multiple times, and errata should be avoiding multiple
submissions

Sandy: am curious about the wiki system that people mentioned.  Is the
idea to have some type of system that tells users whether the fix is
really important?  There are some technical errata that could indicate
implementation problems, and those are the ones most important.   This
team is comfortable with users making their own decisions on what’s
important?

Ted: yes, but it’s a comment system that isn’t representative of
consensus.  And yes, that’s what has to happen (that users make their
own decisions).

Robert: If you have a commenting system, what keeps it from becoming a
replacement for the WG lists, esp the recently published documents? 
What keeps it from becoming an extension of WG fights?  Example:
Pervasive Monitoring BCP.

Ted: This might make the mailing list comments more accessible, because
it would take the place of the mailing lists.  That might be
beneficial.  The useful things would be modded up heavily and would be
the first things to show up.  And there could be interventions if there
were abuses in the system (but that implies someone is monitoring the
system).  This is a system that will be watched by the IETF and they
will bring it to the IESG attention.



** HF to send out raw notes and set up a doodle poll for at least one
more call
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJUwVhQAAoJEER/xjINbZoGoHAH/iA7q1MdREbVZNHahVvXLjj8
Q7H6wqaX2byY1Qe/wmyX6x5/cFO82Eegqr5dg2/6MOr943HZf+500qcgD7MTMBI+
Gwd/yv7ry1D8XChcKzSzUr2gezrnVEux7HurIhDFpHbDaHZNfoxUwxv4ljifgHIR
XJRXuyxYydJRJxttoMEeAz/HOCptv+XdKavlB41kWV1o24avp0uOe6JjNwjVXk5x
I/I9Ql3UPIsx4wFYQjWazx3zOFBSPR+ySUq/hnSGUBZzNqmaEMsct7798EpY+iNF
rR0Juamdr0Pm0dfiL2no5o2SU0s7SCOgPxFASSrx48xhR8BfF0Eq1d4t9+MbG6s=
=HA1c
-----END PGP SIGNATURE-----