Re: [Techspec] Notes from Techspec

[Leslie Daigle <[e-mail]>]

Spencer,

Thanks again for taking notes from the session!

I've uploaded the HTML version as (draft) minutes.  I note
that there seem to be some last names missing.  Are there
any updates people can provide before I post
final minutes?


(FYI -- see the HTML-formatted version at

http://www3.ietf.org/proceedings/05nov/minutes/techspec.html

  -- it's easier to read the sub-bulleting).

Leslie.


Spencer Dawkins wrote:
> My notes follow - please let me know if you have corrections for the 
> proceedings.
> 
> Thanks!
> 
> Spencer
> 
> BoF: TechSpec -- Requirements for IETF Technical Specification Publication
> 
> Chair: Leslie Daigle <[e-mail]>
> Notes: Spencer Dawkins <[e-mail]>
> Mailing list: [e-mail]
> 
> Agenda
> ------
> 
>  a.. <we actually need jabber monitors more than we need jabber scribes, 
> these days?.
> 
> Overview & Introduction [BoF Chair]
> 
>  a.. BOF description didn't pass through an editor, of course...
> 
>  b.. Want to enumerate IETF publication requirements (BCP or otherwise)
>  c.. See list of "what we are not doing" in the BOF description
>  d.. Bob Braden - document formats should be discussed (somewhere, and 
> why not here?) Document lifecycle should also include the part where 
> people actually read documents?
> ?. Update from experiment in early copy editing [Bert Wijnen, RFC Editor]
> 
>  a.. ops.ietf.org/ece for documents that have gone through the 
> experiment, description, etc.
>  b.. Initiated by IAB, IESG, RFC-Editor
>  c.. Alice Hagens did the actual work...
>  d.. WGs participating are from OPS, TSV, SEC, and documents include a MIB
>  e.. This was a very, very, very limited initial experiment with six 
> documents - need to draw conclusions carefully
>  f.. Hoping for positive impact on following steps, less copy-editing, 
> shortened AUTH-48
>  g.. Experiment does not mean documents have priority in queue
>  h.. We have documents in AUTH-48 for six months now, this is not good, 
> and changes cause churn late in the process
>  i.. Bert serialized the experiment inputs/outputs to track timings, 
> number of changes, etc.
> 
>  j.. RFC Editor committed to one-week turnaround
>  k.. Using XML as format for the experiment
>  l.. Average data points - 3 days at RFC Editor, 1 hour per 8/9 pages by 
> RFC-Editor, 1149 changed lines, 1360 changes by authors after the 
> authors got documents back (311 were for verb tense, 860 caused by 
> author error submitting wrong version of document)
>  m.. Still need to check number of changes after WGLC (only one doc has 
> completed WGLC)
>  n.. We skipped the source control steps, may need an IETF service for this
>  o.. Positive experience, great turnaround, cannot serialize in 
> production, need to be careful what gets shipped in each direction, need 
> to follow what happens in the rest of the process
>  p.. Next steps - follow up, do more experiments, figure out how to 
> un-serialize
>  q.. Aaron - "commodity copy-editors" - RFC Editor is using these now. 
> Not IETF people, maybe not even computer scientists. They do lots of 
> markups, but these get filtered before going to the authors. Alice has 
> done both tasks in this experiment - think about what happens when 
> authors interact with commidity copy-editors, who may be handing you 
> changes on a sheet of paper and not new XML, for instance.
>  r.. Pete - commodity copy editors may make a lot more changes and slow 
> down the process?
> 
>  s.. Aaron - "yes", but even technical editors aren't familiar with 
> document structure at the IETF
>  t.. Bob - copy editors make stylistic changes, RFC Editor has been 
> discouraged from doing this - this is "early editing", not "early 
> copy-editing" - "agreement" substituted for "consensus"
> Alice - take on experiment
> 
>  a.. Using paper, then entering edits into XML, add questions for 
> authors and major revisions
>  b.. Differences between documents - we only edit description clauses in 
> MIBs, but in other documents we're figuring out how to do VSPACE and 
> even looking at the wrong version of the document. 8-9 pages/hour seems 
> typical
>  c.. Patrik - VSPACE in lists is context-dependent - do you summarize 
> problems you encounter with the tools? If Alice doesn't think the 
> problem is learning experience..
>  d.. Need to do some additional measures from RFC Editor perspective - 
> time/changes, issues not resolved, AUTH48 progress
> Miguel Garcia - editor for Diameter application for SIP draft
> 
>  a.. Document is 80 pages
>  b.. Did two formal revisions over about 30 days
>  c.. Some suggestions result in a large number of changed lines 
> (inconsistent terminology, for example)
>  d.. Still doing some changes at the end of the process (some technical, 
> some formatting, etc.)
>  e.. All suggestions were editorial, not technical
>  f.. Very positive experience, especially not making global changes in 
> AUTH48 with OLD/NEW
>  g.. WG always had opportunity to see changes
>  h.. No slowdown detected
>  i.. Some documents have lots of dependencies - this may be a drawback 
> for something like GRUU
> Elwyn - editor for NAT-PT-to-Experimental draft
> 
>  a.. Had actually been through WGLC, perhaps this was a misunderstanding
>  b.. Did about a three-day cycle, with lots of small changes ("which" 
> instead of "that", commas). Almost all changes were editorial, did 
> identify terminology overload and one other non-editorial issue
>  c.. One "false positive" editorially, but we fixed the sentence in a 
> better way anyhow
>  d.. Resulting document is a lot better when it hits IETF LC (with 
> Gen-ART reviewing hat on)
>  e.. Not sure how much parallelization is possible
>  f.. Bert - can't wait six months for a doc, can't pay $5M, it's a tradeoff
>  g.. Allison - working group text is stable makes it better - what 
> happens if the WG is still making lots of diffs? WGLC is when people 
> read documents, need to take this into account
>  h.. Bert - this document went through WGLC and was still difficult to 
> read - process helped
> 
> Jari - co-chair of MOBIKE
>  a.. Also went quite smoothly, had to wait in queue a little
>  b.. Focus resources on documents that DO need the changes
>  c.. Elwyn - would be easy for copy editors to do this, Gen-ART isn't 
> copy editors
>  d.. Bob - "some documents need few changes, therefore there should be 
> no changes?"
> Discussion
> 
>  a.. Spencer - Really like opportunity for global suggestions "early". 
> Matches my Gen-ART experience - maybe experiment with one or two 
> documents BEFORE WGLC? and sets of documents need to be consistent 
> internally, too
> 
>  b.. David Black - Also Gen-ART - copy editors can clean up language, 
> not technical lack of clarity
>  c.. RFC Editor - global changes in AUTH48 - prefer NEW/OLD because many 
> douments have sets of similar text, want to know exactly where the text 
> is - and we keep sets of documents with one copy editor
> . 3. Review and discussion of draft-mankin-pub-req [Allison Mankin]
> 
>  a.. Have never really thought about document lifetime in the IETF 
> before - we pick up readers early because of "running code", so adding 
> readers to the lifecycle isn't easy to think about
>  b.. Bob - at least add readers to the diagram?
>  c.. (Req-3) - Detailed visibility into Tech Pub tracking
>  d.. Aaron - what does Req-3 actually mean? for example? Correspondence 
> with editors visible to WG, like correspondence with ADs - copying the 
> WG mailing list would be sufficient?
>  e.. Brian - to clarify the clarification - we are working toward 
> interaction visibility with IESG, don't do as we do, do as we say
>  f.. Aaron - want to know what the RFC Editor does now?
>  g.. Leslie - yes, but not in this meeting, we're focused on what our 
> requirements are first
>  h.. Allison - are we ready to talk about adopting these requirements yet?
>  i.. Bob - RFC Editor is part of the community and agrees with Req-2 and 
> Req-3 strongly - need to think about resources required
>  j.. Bert - sort of agree, but first requirement is that we hit a button 
> and the document appears (applause in the room). How much time are we 
> prepared to accept? We don't need detailed visiblity if the document 
> appears in two weeks.
>  k.. Leslie - we can move work, but it doesn't go away, and if we need 
> visibility, we need it no matter where the work is
>  l.. (Req-2) - do we want seamless tracking of document movement into 
> Tech Pub?
>  m.. Bert - why not one tracking system?
>  n.. Eric - do we need detailed visibility of line-by-line editing? May 
> not be appropriate
>  o.. (Pre-Req-7) One coordinator for the editor process?
>  p.. Bob - what's the issue here?
>  q.. Allison - dealing with five authors, or one of them, or ...
>  r.. Bob - we contact all authors to make sure they all exist and are 
> still alive - want us to change our policy?
>  s.. Leslie - have been on IAB telechats for years - there is lack of 
> clarity about who is doing what
>  t.. Pekka - sometimes one author isn't responsive, and it's not clear 
> who steps up when that happens
>  u.. Thomas - yes, we want to have one person. The current system works 
> pretty well most of the time. We don't really have authors anymore with 
> WG text, we have WG editors and the other names shouldn't be slowing 
> down the process
>  v.. Scott - no-brainer that either authors or ADs should pick one 
> contact. Final signoff still needs to be by each author
>  w.. Leslie - IETF decision for this is a requirement, need to have 
> discussion about what we decide
>  x.. Margaret - mix together concepts of credit and control. If an 
> author dies, they deserve credit but don't have control (except maybe 
> Bert, who will come back to haunt us). This is the extreme case, not the 
> only case. Steve Deering deserves IPv6 credit but not IPv6 control, at 
> this point. Don't know whose names go on the front page, and maybe we 
> shouldn't even put names on the first page.
>  y.. David - moving a lot of docs through this process, what Scott said 
> was correct (masthead signs off). Proto shepherd is the right victim, 
> and should be able to exempt and remove authors
>  z.. Bob - more comfortable with ADs making this decision - is "DOES" in 
> requirement MUST, MAY, or SHOULD? Margaret is right - first page is 
> tending toward "control", with acknowledgements in a separate section.
> 
>  aa.. (Pre-Req-8) Non-author editing affecting consensus wording
>  ab.. Bob - what happens if consensus wording is grammatically incorrect?
>  ac.. Allison - discuss bigger changes on WG mailing list
>  ad.. Leslie - when non-author editing affects wording, we need to 
> resolve this
>  ae.. (Req-9) No style changes?
> 
>  af.. (Req-10) Small technical changes submitted in a narrow window 
> following approval, not at time of publication
> 
>  ag.. Document shepard and AD can filter these
>  ah.. Jari - queuing takes time, things happen, people implement after 
> approval. These changes show up at AUTH48.
>  ai.. Margaret - after approval, something comes up - what should 
> happen? We don't know what to do when something comes up. WG should 
> decide how to handle large flaws. We have pulled documents out of editor 
> queue - need to know what to do, in a process sense.
>  aj.. Allison - WGs are terrible at saying "no". Trust the shepherd to 
> determine what's a show-stopper? We have finished the process and are 
> collecting issues towards a -bis version. Ask the working group "broken 
> beyond repair?" - don't go through process multiple times for small things.
>  ak.. Leslie - need to socialize our process from multiple perspectives, 
> but can't do that in next 43 minutes. Need to discuss on the list, need 
> to move on now. We will revise the document - we know that. Discuss 
> creating a working group?
>  al.. Bob - document is scattershot
>  am.. Allison - someone else will be holding the pen
>  an.. Bob - needs to address more issues and provide more context - not 
> the basis for a working group yet
>  ao.. John - this document is part of our game of "whack-a-mole" - we're 
> off to the land of requirements, not solutions. This topic is worth 
> working on, this document is not the right way to go
>  ap.. Scott - there is a pony underneath the poop someplace. Some of 
> this document is OK, but it's the problem set, not the document.
>  aq.. Dan Burnett - process question - also need to know who is going to 
> do the work
>  ar.. Bert - yes, we combined both questions (should work on it, will 
> work on it). A lot of this is darned complex. What happens before IESG 
> approval is a lot cheaper than what happens after IESG approval, and 
> part of the cost is delay.
>  as.. Margaret - I don't have enough information (not in evidence). Lots 
> of interaction with other short-term efforts. Do short-term efforts, at 
> least in parallel.
> 
>  at.. Leslie - if we are doing competitive bids, we need to know what 
> we're asking for
>  au.. Margaret - we paid $800K based on an SOW, do we need more?
>  av.. Leslie - SOW is what tasks are carried out, not what we WANT 
> carried out
>  aw.. Brian - What Leslie said, plus - need general description text 
> that we don't have in the SOW today
>  ax.. Bob, as member of community - please focus on readership! What an 
> implementor has to focus on to implement anything. That was supposed to 
> happen in Newtrk, but it's not. It's a big elephant.
>  ay.. Allison - what the document is supposed to do, is figure out what 
> we really want, not just what we do, document by document, area by area, 
> with no consistency. We are trying to guide IASA, and they don't know 
> what we want them to do with their dollars
>  az.. Leslie - need to reel all this back in ... for further work, we 
> need more information about purpose, which is known as a charter in 
> these parts
>  ba.. Bert - not trying to pick on our problems, we are putting forth 
> requirements for fixing problems, not for producing documents
>  bb.. David - consistently publishing bug fixes, before and after 
> approval, is a requirement
>  bc.. Margaret - we have substantial requirements for publication, not 
> just for editing (archival quality, etc). In this document, or elsewhere?
>  bd.. Allison - "Cataloging" in the slide deck (which we didn't get to).
>  be.. Thomas - must attach errata to the document in an obvious way
>  bf.. Sam - we need a low-overhead publication path - as part of what 
> the IETF needs. We are trying to raise the bar for what goes into a 
> working group - excluding work that needs to be published.
>  bg.. Harald - I'm worried (perhaps not for the first time, but) - 
> technical publication means getting documents to the public so they can 
> be referenced quickly. Need to know who to talk to in order to do 
> something - open process - and how violently to talk to them. We have 
> major issues with getting documents out, and making sure what comes out 
> is what we approved. Two years to make up our minds isn't part of the 
> process - don't use tech publishing as a holding queue
>  bh.. Pasi - Errata - if our documents took two weeks to publication, we 
> wouldn't need errata
>  bi.. Allison - find a problem a year later, don't want to publish a new 
> RFC to correct an RFC
>  bj.. Pasi - if it was easy and fast to publish a new RFC...
>  bk.. Leslie - interesting... counterpoint is that discussion also shows 
> things down
>  bl.. Bob - the less editing you do, the more errata you publish. One 
> guy in Germany reads all the RFCs and sends in "by the way, have you 
> noticed?" - often up to 10 or so per RFC
>  bm.. Eric - why focus on errors that appear in RFCs? Knuth has an error 
> on every third page and he's one of the most careful authors ever. We 
> have to live with this
>  bn.. Leslie - please continue discussion on the mailing list.
> 
> 5. Determination of where from here for requirements documentation.
> 
>  a.. (did we actually do this?)
> 
> 
> 
> _______________________________________________
> Techspec mailing list
> [e-mail]
> https://www1.ietf.org/mailman/listinfo/techspec
> 

_______________________________________________
Techspec mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/techspec