[netmod] perfect extraction, tabs, and long lines - oh my

[Kent Watsen <[e-mail]>]

[new subject line]

It is one thing for an editor to use tabs during the creation of text,
and another to publish text with an expectation that consumers will
render the tabs the same way.  Either the source editor converts tabs
to spaces, which is interoperable today, or keep the tabs while
publishing metadata in the text, using some TBD standard, enabling
consumers to use the same tab stops.  

If there were a standard enabling the publishing of text including
tabs, it should work for all artwork, not just artwork that has been
folded.  This is similar to the discussion we had before about having
begin/end markers enabling perfect extractions, in that it is also
something that pertains to all artwork, not just artwork that has 
been folded.  

Thus, there are a total of three problems:
  P1: perfect extraction
  P2: tabs
  P3: long lines

Assuming all thing were solved problems, and assuming that we always
want perfect extraction, the possible combinations for the occurrence
of the other two problems are:
  - no tabs or long lines
  - tabs, but no long lines
  - long lines, but no tabs
  - tabs and long lines

How are they ordered?  Clearly supporting perfect extractions has 
to be the outermost thing, but what about the other two?   Does it
matter?

Thinking about solutions:

 - the solution for long-lines is to use a header (not a footer)
   because it's believed important to prime readers *before* they
   read the text.

 - the solution for perfect-extraction could be either:
     - use both a header-and-footer marker (low tech)
     - or use either a header or a footer that encodes
       something like a "num lines" value into the 
       marker.  (note: footer-only okay since the marker
       is for programmatic processors, not the readers)

 - the solution for tabs could be to use either a header
   or a footer that encodes the tab- stop metadata. (note:
   footer-only okay since the marker is for programmatic
   processors, not the readers)


If tabs were to be supported by the folding solution (note: it
doesn't make sense to talk about "folds being supporting by the
tabbing solution"), then either:

  a) tabs are handled *before* folding, and the folding-solution 
     is aware of the tab-solution (i.e., it is able to process 
     the metadata).

      - everybody nods ;)

  b) the folding-solution is really a folding+tab solution, that is,
     it has a built-in way of handling tabs (i.e., encoding tab stop
     metadata) independent of how tabs are handled for text that has
     not been folded.

      - this may be technically possible, but we should avoid having
        two solutions to solve the tab problem.  We would be better
        off solving the tab-problem directly and then use (a).

  c) the folding-solution folds using the source tab stops, but does
     not itself encode metadata about the tab stops, assuming that
     there is a "promise" that the encoding of the metadata will
     occur in a wrapper layer around it.

      - this feels icky, but it seems viable and, would possible
        allow us to proceed with this draft without having to solve
        the tabbing problem now.


Options:

  1) RFC disallows TABS in both the source-input and folded-output.
     ***This is what we currently have***

  2) RFC disallows TABS only in the folded-output, per RFC 7991,
     leaving it to the folding-logic (the script) to decide if it
     wants to:
      a) disallow TABS in the source input (curr script does this)
      b) detect TABS exist and prompt user for TAB stop info
      c) detect TABS and query environment for cur TAB stop info
         (but tab-stops may differ in the shell the text editor,
         or whatever was used to create the text, right?)

  3) RFC allows TABS, and solves it by depending on a tab-solution,
     as described by (a).

  4) RFC allows TABS, but does not solves it, as described by (c).
     This would probably NOT be allowed from a standardization
     perspective.


Moving to (2) would be easy and probably resolves most concerns
here.  

Moving to (3) is possible, but we would do so only to:

 - support non-IETF use cases

 - or pave the way for an rfc7991bis that could depend on the 
   solutions we define here.  

   That is, rfc7991bis could *allow* long-lines and tabs while
   `xml2rfc` applies the solutions being discussed here only
   for when exporting the "plain-text" format (other formats
   may have better ways to support perfect extractions and/or
   not care about long-lines or tabs).

   PS: as a corollary, realize that when we pre-textualizing
       artwork for XML-based submissions, we are somewhat
       worsening the result for other output formats (not
       "plain-text").



Kent