[Techspec] Notes from Techspec

["Spencer Dawkins" <[e-mail]>]

My notes follow - please let me know if you have corrections for the 
proceedings.

Thanks!

Spencer

BoF: TechSpec -- Requirements for IETF Technical Specification Publication

Chair: Leslie Daigle <[e-mail]>
Notes: Spencer Dawkins <[e-mail]>
Mailing list: [e-mail]

Agenda
------

  a.. <we actually need jabber monitors more than we need jabber scribes, 
these days?.

Overview & Introduction [BoF Chair]

  a.. BOF description didn't pass through an editor, of course...

  b.. Want to enumerate IETF publication requirements (BCP or otherwise)
  c.. See list of "what we are not doing" in the BOF description
  d.. Bob Braden - document formats should be discussed (somewhere, and why 
not here?) Document lifecycle should also include the part where people 
actually read documents?
?. Update from experiment in early copy editing [Bert Wijnen, RFC Editor]

  a.. ops.ietf.org/ece for documents that have gone through the experiment, 
description, etc.
  b.. Initiated by IAB, IESG, RFC-Editor
  c.. Alice Hagens did the actual work...
  d.. WGs participating are from OPS, TSV, SEC, and documents include a MIB
  e.. This was a very, very, very limited initial experiment with six 
documents - need to draw conclusions carefully
  f.. Hoping for positive impact on following steps, less copy-editing, 
shortened AUTH-48
  g.. Experiment does not mean documents have priority in queue
  h.. We have documents in AUTH-48 for six months now, this is not good, and 
changes cause churn late in the process
  i.. Bert serialized the experiment inputs/outputs to track timings, number 
of changes, etc.

  j.. RFC Editor committed to one-week turnaround
  k.. Using XML as format for the experiment
  l.. Average data points - 3 days at RFC Editor, 1 hour per 8/9 pages by 
RFC-Editor, 1149 changed lines, 1360 changes by authors after the authors 
got documents back (311 were for verb tense, 860 caused by author error 
submitting wrong version of document)
  m.. Still need to check number of changes after WGLC (only one doc has 
completed WGLC)
  n.. We skipped the source control steps, may need an IETF service for this
  o.. Positive experience, great turnaround, cannot serialize in production, 
need to be careful what gets shipped in each direction, need to follow what 
happens in the rest of the process
  p.. Next steps - follow up, do more experiments, figure out how to 
un-serialize
  q.. Aaron - "commodity copy-editors" - RFC Editor is using these now. Not 
IETF people, maybe not even computer scientists. They do lots of markups, 
but these get filtered before going to the authors. Alice has done both 
tasks in this experiment - think about what happens when authors interact 
with commidity copy-editors, who may be handing you changes on a sheet of 
paper and not new XML, for instance.
  r.. Pete - commodity copy editors may make a lot more changes and slow 
down the process?

  s.. Aaron - "yes", but even technical editors aren't familiar with 
document structure at the IETF
  t.. Bob - copy editors make stylistic changes, RFC Editor has been 
discouraged from doing this - this is "early editing", not "early 
copy-editing" - "agreement" substituted for "consensus"
Alice - take on experiment

  a.. Using paper, then entering edits into XML, add questions for authors 
and major revisions
  b.. Differences between documents - we only edit description clauses in 
MIBs, but in other documents we're figuring out how to do VSPACE and even 
looking at the wrong version of the document. 8-9 pages/hour seems typical
  c.. Patrik - VSPACE in lists is context-dependent - do you summarize 
problems you encounter with the tools? If Alice doesn't think the problem is 
learning experience..
  d.. Need to do some additional measures from RFC Editor perspective - 
time/changes, issues not resolved, AUTH48 progress
Miguel Garcia - editor for Diameter application for SIP draft

  a.. Document is 80 pages
  b.. Did two formal revisions over about 30 days
  c.. Some suggestions result in a large number of changed lines 
(inconsistent terminology, for example)
  d.. Still doing some changes at the end of the process (some technical, 
some formatting, etc.)
  e.. All suggestions were editorial, not technical
  f.. Very positive experience, especially not making global changes in 
AUTH48 with OLD/NEW
  g.. WG always had opportunity to see changes
  h.. No slowdown detected
  i.. Some documents have lots of dependencies - this may be a drawback for 
something like GRUU
Elwyn - editor for NAT-PT-to-Experimental draft

  a.. Had actually been through WGLC, perhaps this was a misunderstanding
  b.. Did about a three-day cycle, with lots of small changes ("which" 
instead of "that", commas). Almost all changes were editorial, did identify 
terminology overload and one other non-editorial issue
  c.. One "false positive" editorially, but we fixed the sentence in a 
better way anyhow
  d.. Resulting document is a lot better when it hits IETF LC (with Gen-ART 
reviewing hat on)
  e.. Not sure how much parallelization is possible
  f.. Bert - can't wait six months for a doc, can't pay $5M, it's a tradeoff
  g.. Allison - working group text is stable makes it better - what happens 
if the WG is still making lots of diffs? WGLC is when people read documents, 
need to take this into account
  h.. Bert - this document went through WGLC and was still difficult to 
read - process helped

Jari - co-chair of MOBIKE
  a.. Also went quite smoothly, had to wait in queue a little
  b.. Focus resources on documents that DO need the changes
  c.. Elwyn - would be easy for copy editors to do this, Gen-ART isn't copy 
editors
  d.. Bob - "some documents need few changes, therefore there should be no 
changes?"
Discussion

  a.. Spencer - Really like opportunity for global suggestions "early". 
Matches my Gen-ART experience - maybe experiment with one or two documents 
BEFORE WGLC? and sets of documents need to be consistent internally, too

  b.. David Black - Also Gen-ART - copy editors can clean up language, not 
technical lack of clarity
  c.. RFC Editor - global changes in AUTH48 - prefer NEW/OLD because many 
douments have sets of similar text, want to know exactly where the text is - 
and we keep sets of documents with one copy editor
. 3. Review and discussion of draft-mankin-pub-req [Allison Mankin]

  a.. Have never really thought about document lifetime in the IETF before - 
we pick up readers early because of "running code", so adding readers to the 
lifecycle isn't easy to think about
  b.. Bob - at least add readers to the diagram?
  c.. (Req-3) - Detailed visibility into Tech Pub tracking
  d.. Aaron - what does Req-3 actually mean? for example? Correspondence 
with editors visible to WG, like correspondence with ADs - copying the WG 
mailing list would be sufficient?
  e.. Brian - to clarify the clarification - we are working toward 
interaction visibility with IESG, don't do as we do, do as we say
  f.. Aaron - want to know what the RFC Editor does now?
  g.. Leslie - yes, but not in this meeting, we're focused on what our 
requirements are first
  h.. Allison - are we ready to talk about adopting these requirements yet?
  i.. Bob - RFC Editor is part of the community and agrees with Req-2 and 
Req-3 strongly - need to think about resources required
  j.. Bert - sort of agree, but first requirement is that we hit a button 
and the document appears (applause in the room). How much time are we 
prepared to accept? We don't need detailed visiblity if the document appears 
in two weeks.
  k.. Leslie - we can move work, but it doesn't go away, and if we need 
visibility, we need it no matter where the work is
  l.. (Req-2) - do we want seamless tracking of document movement into Tech 
Pub?
  m.. Bert - why not one tracking system?
  n.. Eric - do we need detailed visibility of line-by-line editing? May not 
be appropriate
  o.. (Pre-Req-7) One coordinator for the editor process?
  p.. Bob - what's the issue here?
  q.. Allison - dealing with five authors, or one of them, or ...
  r.. Bob - we contact all authors to make sure they all exist and are still 
alive - want us to change our policy?
  s.. Leslie - have been on IAB telechats for years - there is lack of 
clarity about who is doing what
  t.. Pekka - sometimes one author isn't responsive, and it's not clear who 
steps up when that happens
  u.. Thomas - yes, we want to have one person. The current system works 
pretty well most of the time. We don't really have authors anymore with WG 
text, we have WG editors and the other names shouldn't be slowing down the 
process
  v.. Scott - no-brainer that either authors or ADs should pick one contact. 
Final signoff still needs to be by each author
  w.. Leslie - IETF decision for this is a requirement, need to have 
discussion about what we decide
  x.. Margaret - mix together concepts of credit and control. If an author 
dies, they deserve credit but don't have control (except maybe Bert, who 
will come back to haunt us). This is the extreme case, not the only case. 
Steve Deering deserves IPv6 credit but not IPv6 control, at this point. 
Don't know whose names go on the front page, and maybe we shouldn't even put 
names on the first page.
  y.. David - moving a lot of docs through this process, what Scott said was 
correct (masthead signs off). Proto shepherd is the right victim, and should 
be able to exempt and remove authors
  z.. Bob - more comfortable with ADs making this decision - is "DOES" in 
requirement MUST, MAY, or SHOULD? Margaret is right - first page is tending 
toward "control", with acknowledgements in a separate section.

  aa.. (Pre-Req-8) Non-author editing affecting consensus wording
  ab.. Bob - what happens if consensus wording is grammatically incorrect?
  ac.. Allison - discuss bigger changes on WG mailing list
  ad.. Leslie - when non-author editing affects wording, we need to resolve 
this
  ae.. (Req-9) No style changes?

  af.. (Req-10) Small technical changes submitted in a narrow window 
following approval, not at time of publication

  ag.. Document shepard and AD can filter these
  ah.. Jari - queuing takes time, things happen, people implement after 
approval. These changes show up at AUTH48.
  ai.. Margaret - after approval, something comes up - what should happen? 
We don't know what to do when something comes up. WG should decide how to 
handle large flaws. We have pulled documents out of editor queue - need to 
know what to do, in a process sense.
  aj.. Allison - WGs are terrible at saying "no". Trust the shepherd to 
determine what's a show-stopper? We have finished the process and are 
collecting issues towards a -bis version. Ask the working group "broken 
beyond repair?" - don't go through process multiple times for small things.
  ak.. Leslie - need to socialize our process from multiple perspectives, 
but can't do that in next 43 minutes. Need to discuss on the list, need to 
move on now. We will revise the document - we know that. Discuss creating a 
working group?
  al.. Bob - document is scattershot
  am.. Allison - someone else will be holding the pen
  an.. Bob - needs to address more issues and provide more context - not the 
basis for a working group yet
  ao.. John - this document is part of our game of "whack-a-mole" - we're 
off to the land of requirements, not solutions. This topic is worth working 
on, this document is not the right way to go
  ap.. Scott - there is a pony underneath the poop someplace. Some of this 
document is OK, but it's the problem set, not the document.
  aq.. Dan Burnett - process question - also need to know who is going to do 
the work
  ar.. Bert - yes, we combined both questions (should work on it, will work 
on it). A lot of this is darned complex. What happens before IESG approval 
is a lot cheaper than what happens after IESG approval, and part of the cost 
is delay.
  as.. Margaret - I don't have enough information (not in evidence). Lots of 
interaction with other short-term efforts. Do short-term efforts, at least 
in parallel.

  at.. Leslie - if we are doing competitive bids, we need to know what we're 
asking for
  au.. Margaret - we paid $800K based on an SOW, do we need more?
  av.. Leslie - SOW is what tasks are carried out, not what we WANT carried 
out
  aw.. Brian - What Leslie said, plus - need general description text that 
we don't have in the SOW today
  ax.. Bob, as member of community - please focus on readership! What an 
implementor has to focus on to implement anything. That was supposed to 
happen in Newtrk, but it's not. It's a big elephant.
  ay.. Allison - what the document is supposed to do, is figure out what we 
really want, not just what we do, document by document, area by area, with 
no consistency. We are trying to guide IASA, and they don't know what we 
want them to do with their dollars
  az.. Leslie - need to reel all this back in ... for further work, we need 
more information about purpose, which is known as a charter in these parts
  ba.. Bert - not trying to pick on our problems, we are putting forth 
requirements for fixing problems, not for producing documents
  bb.. David - consistently publishing bug fixes, before and after approval, 
is a requirement
  bc.. Margaret - we have substantial requirements for publication, not just 
for editing (archival quality, etc). In this document, or elsewhere?
  bd.. Allison - "Cataloging" in the slide deck (which we didn't get to).
  be.. Thomas - must attach errata to the document in an obvious way
  bf.. Sam - we need a low-overhead publication path - as part of what the 
IETF needs. We are trying to raise the bar for what goes into a working 
group - excluding work that needs to be published.
  bg.. Harald - I'm worried (perhaps not for the first time, but) - 
technical publication means getting documents to the public so they can be 
referenced quickly. Need to know who to talk to in order to do something - 
open process - and how violently to talk to them. We have major issues with 
getting documents out, and making sure what comes out is what we approved. 
Two years to make up our minds isn't part of the process - don't use tech 
publishing as a holding queue
  bh.. Pasi - Errata - if our documents took two weeks to publication, we 
wouldn't need errata
  bi.. Allison - find a problem a year later, don't want to publish a new 
RFC to correct an RFC
  bj.. Pasi - if it was easy and fast to publish a new RFC...
  bk.. Leslie - interesting... counterpoint is that discussion also shows 
things down
  bl.. Bob - the less editing you do, the more errata you publish. One guy 
in Germany reads all the RFCs and sends in "by the way, have you noticed?" - 
often up to 10 or so per RFC
  bm.. Eric - why focus on errors that appear in RFCs? Knuth has an error on 
every third page and he's one of the most careful authors ever. We have to 
live with this
  bn.. Leslie - please continue discussion on the mailing list.

5. Determination of where from here for requirements documentation.

  a.. (did we actually do this?)



_______________________________________________
Techspec mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/techspec