[tcpm] On TCP Urgent processing

[Alfred Hönes <[e-mail]>]

On Wed, 11 Mar 2009 15:56:11 -0500, TCPM co-chair David Borman wrote:

> One of the items on the TCPM agenda for IETF 74 is the TCP Urgent
> pointer. I want this to be a very focused discussion around one
> question: Should the urgent pointer point to the last byte of
> urgent data, or the first byte of non-urgent data?

IMO that is the most worse question to start with.
This question should be posed -- if at all -- *after* exploring
and conceptionally resolving the basic inconsistencies.

Therefore, I very much appreciate the message sent today
as a step forward in a good direction:

> At the TCPM session on Monday, I intend to take a hum on whether
> or not the WG wants to take on the issue of the definition of
> the TCP Urgent pointer (the last byte of urgent data), and its
> contradiction with most TCP implementations (the first byte
> after the urgent data), as documented in
>   draft-gont-tcpm-urgent-data-01.txt.
> It would be nice to get a feel from the mailing list prior
> to the meeting.
>
> Please reply whether you are in favor of, against or have no
>  opinion on adopting this as a WG item.
>
>                   -David Borman, TCPM co-chair
>

I very much support adopting TCP URG clarifications as a work item
for TCPM, and to base that on draft-gont-tcpm-urgent-data-01
as a starting point, but I strongly oppose using the term
"Urgent Data" as a given in this context.
IMO, "Urgent Data" is the term that needs clarification.

The task should be organized in a very different way than
proposed by David.

The utmost first step for any follow-on draft must be to replace
"Urgent Data" by "Urgent Processing" or "Urgent Mode" in the
document title and similarly in the draft name.  See below!


For the moment: Please don't yawl, read on first!


I'd like to use a picture to start explaining my position:

If the fire-brigade is called to fight a conflagration,
it should not ask the question whether the flames are more hot
on the left-hand side of the fire or on the right-hand side,
it should try to extinct the fire; and if the fire is fed by
fermentation gas, the main problem is to get rid of the mould
this gas is continuing to emanate from.

However, the most important task to be undertaken initially
by the fire-brigade would be to rescue the people affected by
the fire !


Quite some time ago, I had started investigations on the background
of the Urgent Processing mess, and studied the historical documents
available online.  This work had been interrupted and resumed later,
but it could not be taken to an exhaustive end, since in the
meantime the RFC Editor has made available a wealth of old IENs;
sadly, these documents are only available as scanned facsimile
copies of mediocre quality, and hence not amenable to `grep`ing
and other text search methods at hand.

David already has explained many of the historical facts in his
recent postings in response to implementer discussions on the list.
In particularly, he has emphasized that there's no concept of
'out-of-band data' in TCP.
This was present in X.25 and the old ARPANET, and in Telnet, and
in the socket API (cf. AF_CCITT functionality!), when TCP has been
specified, and governments and agencies aligned with CCITT and
evolving ISO activities had blindly overtaken the requirement to
support this concept for any 'officially acceptable' Transport
Protocol, and so apparently the 'TCP committee' was forced to
cheet a bit to defeat the pressure they have been faced and get
their proposal socialized.

I will try to get a detailed summary of my results so far posted
to the list by Sunday evening (European time), i.e. early afternoon
Pacific Time, with my conclusions drawn, and more rationale for the
proposal made below -- but IETF-unrelated circumstances might
absorb my cycles.  I hope the details here should suffice to
explain my point of view.


The 'mould' in the picture above, applied to TCP Urgent processing
lies in the term "urgent data".  There are at least a handfull of
different and incompatible definitions of this term buried in
various documents, in the brains of developers, and in
implementations -- both of TCP stacks and of applications.

The basic concept in RFC 793 and all its predecessors since the
introduction of the Urgent Pointer into TCP per RFC 675 (Dec. 1974)
is "Urgent Processing", designating the TCP receiver skipping quickly
over data sent beforehand but having become 'OBE' by some event at
the TCP sender -- typically a human user hitting some "Interrupt" key.

As a Mathematician, I'm used to start with precise definitions
before talking about properties and drawing conclusions.

Thus, I insist on going back to the fundamental terms and not try
to base arguing on fuzzy, derived terms like "Urgent Data".
The two terms precisely defined in the glossary of RFC 793 and its
predecessors are "URG" and "urgent pointer" (RFC 793, pg.84):

| URG
|         A control bit (urgent), occupying no sequence space, used to
|         indicate that the receiving user should be notified to do
|         urgent processing as long as there is data to be consumed with
|         sequence numbers less than the value indicated in the urgent
|         pointer.
|
| urgent pointer
|         A control field meaningful only when the URG bit is on.  This
|         field communicates the value of the urgent pointer which
|         indicates the data octet associated with the sending user's
|         urgent call.

Since these definitions are rather precise, all considerations
and conclusions should take these as the starting point.

IMO, it does not make any sense to talk about the relationship
of the Urgent Pointer and "urgent data" before we have found
consensus on what "urgent data" shall mean, with a definition
solely based on the basic terms and the processing model laid
out in Section 3.7 of RFC 793, on pp. 41/42.

Unfortunately, RFC 793 contains text passages that apparently
introduce the term "urgent data" with three different meanings.
This issue needs to be investigated and resolved *before*
attempting to decide what should be corrected/updated.

For some details and ideas, see below.

The basic reason of why implementations did not follow
the 'corrections' to RFC 793 first published in RFC 924
( not: RFC 961, as indicated in the last paragraph of
  Section 2.2 of draft-gont-tcpm-urgent-data-01 ! )
and finally codified in RFC 1122, seems to be that the
terminology and the processing rules were partially contradictory
from the beginning -- and that apparently was caused by historical
circumstances and pressure exerted on the "TCP committee" over a
long time span in these old days.

Therefore, any attempt to clarify the Urgent Processing in TCP
should thoroughly assess all the contradictions and find a fully
consistent way to describe the intended processing.

Only a fully consistent description and specification has a chance
to become accepted and adopted by implementers in the future.


Now, that's all interesting and very important for future TCP
implementations, but we need to acknowledge the inertia of mass
of the deployed TCP kernel code.

Therefore, returning to the picture painted above, the *first*
step should be to formulate strong recommendations for how
applications and in particular implementations of applications
should make use of TCP Urgent Processing (if any).
This IMO should be based on the concepts explained in the two
definitions from RFC 793 quoted above.

Since decades, Telnet makes use of URG, and apparently this
all works (almost) properly, independent of the underlying
TCP stacks' behavior at the sending and receiving side.
(BTW: the discussion in RFC 1123 seems also to be disturbed.)
The reason seems to be simple: being defensive, and not
assigning semantics to the data delivered with the sending
side's urgent call.
According to RFC 854, the Telnet client sends a DM NVT control
character which is a transparent NOP for the Telnet server.
Thus, it does not matter for the Telnet server how the TCP
stack interprets the Urgent pointer.

I therefore recommend to put on the agenda of a new TCPM work item
on TCP URG, in total comprised of two steps:

(1) Give recommendations for application protocol designers
    and application programmers of how to best use the common
    Sockets API for Urgent Processing, in order to accommodate the
    legacy problems and the inconsistencies in the specifications.

    This should target BCP status, and it should have a tight
    milestone of not more than 6 months for passing to the IESG.

    Most elements of such document already are in
    draft-gont-tspm-urgent-data-01 ;
    my personal expectations for the basic recommendations are:

    o  MUST always use OOBINLINE;
    o  SHOULD only send a single octet in the urgent call;
    o  SHOULD use syntactically transparent data in the urgent call
       (equivalent to DM == NOP in Telnet);
    o  intermediate systems MUST NOT scrub the TCP URG flag
       (URG is an entirely end-to-end indication);
    o  a hint that a parallel TCP connection ('control channel')
       SHOULD be used if an application needs true OOB data transfer.

The second step to be undertaken by the WG should be a thorough
review of the most important specifications (RFC 793 and RFC 1122
for TCP, RFC 854 and RFC 1123 for Telnet) w.r.t. Urgent Processing.

Since so many folks stick with the term "Urgent Data", I painfully
am aware of that my favorite goal, eliding that term from the
specifications, is hopelessly inachievable.

Based on the captured intent of the original specifications,
therefore, a unique definition for "Urgent Data" should be found.
In RFC 793, three possible candidate definitions are sublimely used:

    a) the data 'temporally' ahead of the Urgent Pointer,
       i.e. the all the buffered data that are being invalidated
       by the 'urgent' call (TCP 'SEND' call in the abstract TCP
       'User Interface' of RFC 793, section 3.8, pp. 46/47, with
       URGENT flag set);

    b) the data sent with the 'urgent' call;

    c) the union of both a) and b).

Note #1: Another possible definition -- closely related to b) --,
    d) the "out-of-band-data";
  should not be a candidate for consideration, because TCP is
  based on the idea that maintaining a parallel connection for
  'control' data (if needed at all) is much more efficient and
  cost effective in TCP/IP than the protocol overhead that would
  be needed for OOB data, like it was in X.25; the single, flow-
  controlled byte-stream pipe model was the basic concept for TCP.

Note #2: Apparently, for the most time RFC 793 assumes definition a),
  whereas RFCs 924, 944, 961, 991, 1011, and finally RFC 1122
  assume definition c).  This explains part of the trouble we have.

Note #3: The ambiguity in RFC 793 mostly stems from that it remains
  underspecified whether the urgent pointer refers to the state of
  the sequence number at entry to the event processing or at its
  end, i.e. whether (on pg. 56)  "SND.UP <- SND.NXT - 1" should
  happen before of after processing the data handed over in the
  SEND with URGENT flag set.

Note #4: There is more trouble in RFC 793 related to Urgent mode;
  in particular, the event processing rules deal with URGENT flag
  for the OPEN call whereas the OPEN Interface definition does not
  contain such flag at all.

After that analysis and definition have been done,
the necessary updates for the old specifications to make
tme consistently use the definition can be derived.

Thus, the second delivery of the TCP URG work item should be:

(2)  A Normative (Standards Track) document clarifying the
     definition of the term "Urgent Data" for TCP and Updating
     primarily RFC 793 and RFC 1122, but for consistency also
     RFC 854 and RFC 1123, to make them conformant to the
     clarified definition and provide consistent and self-
     consistent processing rules.

     This should be subject to an ambitious milestone of one year
     until being passed to the IESG.


I will not be able to attend the meeting next week, but ...

I'd offer to work as a co-editor with Fernando Gont for a TCPM
work item shaped along the ideas and guidelines presented above,
but more volunteers to help with completing the investigations
of the TCP-related IEN documents would be heartfully welcome,
in order to give the whole effort the best possible foundation
to be in the spirit of the original designers of TCP.


Kind regards,
  Alfred Hönes.

-- 

+------------------------+--------------------------------------------+
| TR-Sys Alfred Hoenes   |  Alfred Hoenes   Dipl.-Math., Dipl.-Phys.  |
| Gerlinger Strasse 12   |  Phone: [phone], Fax: -18         |
| D-71254  Ditzingen     |  E-Mail:  [e-mail]                     |
+------------------------+--------------------------------------------+