[Errata-design] Untidy notes from today's call - 2 February 2015

["Heather Flanagan (RFC Series Editor)" <[e-mail]>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Attending:
Heather, Sandy, Barry, Pete, Ted, Stephen, Robert

Apologies:
Nevil, Matt, Joel

Summary of last call:
Mostly discussed the first topic, measures for a successful errata
system.  Measures included:
* differentiate between levels of usefulness for an implementer/reader
visually and through metadata
* or, possibly another way to say that, more clearly differentiates
between editorial and technical things implementors may need to care about
* shouldn’t be a PITA to use

Other critical points:
* RPC to be the gatekeeper for editorial errata; need to have a way to
pass through things that look editorial but may have impact on the
technical meaning of the text.
* level of effort is a consideration, both on the part of the RPC and
the community, but it is only one criteria.  
* editorial errata and EFL issues may tie together, but that’s hard for
this group to really understand without more information
* a system that allows for modding of technical reports seems to have
consensus

Any more discussion that folks want to close out from the last call?

Pete: question for Stephen - with regards to simple editorial stuff, the
key is that we shouldn’t be bothered by it, but you don’t care if the
RPC tracks those?

Stephen: Yes.  We’ll need to work our way through what “too big a
change” may be, but for the more simple things, that’s ok.  And they
need to do anti-spam.

Next topic: existing proposals
    a. Replace the concept of errata with a system that would
        allow anyone to comment on an RFC, with the RFC Editor
        or stream controllers to mark some concepts as important.
        The 'errata system' would be a system of comments, not
        a system of discretely managed items. Comments that
        stream controllers (or their delegates) mark as important
        could then be shown through an “errata” link. Alternatively,
        there could be a general 'mod up/down' [1] system that
        anyone with a datatracker account could use to indicate the
        importance of the entry. The RFC Editor could handle
        unambiguous typos.

(This one is basically the modding system)

   b. [A]n IETF-operated web site that is preferable to the
       competition (e.g., tools) and that presents errata of this sort as
       highlighted in a different color on the presented RFC, rather than
       as comments that are in a separate visual stream from the
       presented RFC. I would like errata to be done similarly, if they
       are verified by someone who can check whether a new
       consensus is required to affirm the erratum.

Ted: the idea is that we have something as user friendly and readable as
tools.ietf.org.  The presentation of rfc on tools is more usable than
those on datatracker.ietf.org.  On top of that, show if there are
editorial changes that have been accepted, to show those with the RFC as
it is presented, so you don’t have to go hunting somewhere else to find
errata of that sort.  Then, the proposal that is mentioned in (a) would
also link off this space.  Would like to SEO the website so that people
go to this corrected location.  Doesn’t intrinsically care if this is
the ietf.org site or the rfc-editor.org site, as long as it lands in a
consistent place and shows the corrections on the fly.

Robert: For this team, it is a good question to talk about the threshold
for change should be for a published RFC.  Should things render so there
is a visual hiccup to highlight the change?

Ted: If it is strictly editorial, would like to see the change in the
flow of the text, but possibly with a change in color.

Heather: This may result in an interesting audit trail for when Sandy
has to respond in a subpoena.

Pete: As long as it’s just an overlay, it shouldn’t be a problem.

Sandy: Concerned about where to draw the line about what is a simple
editorial correction and how many corrections can go in.  Also, did hear
the concerns about how readability helps EFL speakers, but concerned
about why there is so much focus on making a new version of the
document?  This could require a considerable amount of work, and yet
there is so much less focus on the tech portion.  Why?

Pete: Talking only technical at this point, mod up/down, if there is a
technical correction that people really must notice, there is going to
be a point in which during the discussion, an event where enough people
have modded up the comment, it goes into a barrel that it must be
displayed with the text.  Whether that is as a pop up, or changed text
(highlighted red), doesn’t matter.  This distinguishes it from the
majority of errata today that really doesn’t rise to that level, and the
RPC should not be involved in those discussions.  Those don’t need to be
shown as a correction.

Robert: Replay in your head what you hand to the lawyer when you change
the content due to technical issue.

Pete: Because that could be meaningful in a subpoena.

Sandy: that big technical change is where I would prefer that there is
an updated document with the correction in line, rather than an
editorial nit that most people will gloss over.

Ted: important to pay attention to a few things.  We’re not talking
about technical changes because the vast majority of these changes would
not be appropriate to apply to the RFC, even if the change was correct,
because it goes against the consensus that created the document.  As far
as the text/nits go, the example we’re using is far more important to
get right than editorial nits.  But examples of that type of erratum are
very rare.  

Pete: in those rare cases where you have the 500 vs 5000 problem, the
expectation is that it will be fixed with a new document eventually, but
it’s too important to just wait around for that to happen.  We’re not
changing WG consensus, we’re correcting a very important implementation
error.

Sandy: Is having a new version of the RFC to address editorial nits
worth the effort it will take to do it?

Ted: If the submitter is the one that has to do the work, and that they
have to submit it as a simple correction for the RPC to simply click
“yes or no”.  Or it will be something non-obvious, at which point you
won’t accept it.

Robert: It’s more a question of how to implement this.  

Ted: there is a tooling question here.

Sandy: The RPC will still have more to do here, and it’s still
questionable as to whether it’s worth the staffing resources to do this.

Ted: the a/b test should answer that problem.  The a is the original
copy, and the b is the correction.  The differences should be something
that doesn’t require a lot of thinking.  And as far as the backlog, no,
the RPC shouldn’t deal with it.  We put it out there for people to
consider whether they want to resubmit the errata in the new system.

Barry: If you want to do it that way, as long as people’s names are
attached to the errata, people will do it. They want the credit.  So we
need to have a way to have the name recorded, but not show up to the
public.  We don’t want that to be a motivation.

Stephen: If people agree than an approach here is to find a solution is
good enough, then we should just start experimenting with tools. 
(general agreement).  It will be worth, along this process, for the RPC
and Tools to talk about whether there will still be multiple
representations of RFCs across different sites, and depending on what
Tool system we pick, if it is under laid by a source control system,
that may be straightforward to create something for a court case as
evidence.  Example: git submits are signed.

Robert: People point to gitlab to get what they want out of github, and
we should run that experiment

Heather: I still want to see a diagram of what this would all look like,
and could use pointers on how we run an experiment with tools.

Robert: an experiment would require knowing what we’re wanting to play
with.  We need the list of possible tools, we need some disposable
instances to play with within this team.  We need to build out the list
of the things we should try.

Ted: We’re not going to be happy with just git’s representation of diff
between two XML files.

Sandy: That raised a separate question - when the errata is submitted,
in the future format era, what is it getting filed against?

Ted: the XML

Stephen: Or a particular rendering.

Heather: We do need to track those bugs, but they are tool bugs.  Still,
users won’t know the difference.

Stephen: Could this be done along side the code sprint?  (a lot of
conflict with participants, but may be possible)

Heather: team needs to send to the list what tool(s) they want to try
out.  HF will

Robert: we need to start capturing the issues and particularly rough
edges that we decided not to jump into (e.g., what the errata is against
(rendered formats, XML))

Stephen: Can we just use an etherpad somewhere?  (sure, as long as it
gets tracked)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJU0BUGAAoJEER/xjINbZoGkL8H/i3Tl9mNLHIg9yeygrMCcY76
odEiuZIEXwiZ1TP4ZsMHBUdbi+YfE5fPqQLV4nORWQc9vJKBZoCNe1qKqlMw4dl9
UJqF0/R4Eb2Iy9Jz6FzYPeaGxvI8P8jbdLIBLPKC2dKhp9aS6XN9N52KbONur9Af
0l1zcyOUn9VGV+HDBEeBfFAuDiEGdl7i3RV9dbL5Iw6+eeRewX4hCMo6IZgsh3FU
9kR7I0qzhHiht4oo3IcKNlRE/s8X0nuYrQ2YWhKczJPJa/AtFDjCyS2dN1/gXsWt
ulGJFmFfIataE4y7LZQ+wCyoBS6VleIeaQtwD9D9e4q7CGxlfJpNiG1kzs+I/K0=
=P4ZH
-----END PGP SIGNATURE-----