Re: [tcpm] On TCP Urgent processing

[Alfred Hönes <[e-mail]>]

Fernando,
thanks for your response.

I assume that engaging in this discussion is more important now
than consolidating the reference material I promised to present,
so please see my elaborations in-line below.


> Alfred Hönes wrote:
>
>> The utmost first step for any follow-on draft must be to replace
>> "Urgent Data" by "Urgent Processing" or "Urgent Mode" in the
>> document title and similarly in the draft name.  See below!
>
> Well, this one shouldn't be a problem. I can live with it. :-)
>
>
>> The basic reason of why implementations did not follow
>> the 'corrections' to RFC 793 first published in RFC 924
>> ( not: RFC 961, as indicated in the last paragraph of
>>   Section 2.2 of draft-gont-tcpm-urgent-data-01 ! )
>> and finally codified in RFC 1122, seems to be that the
>> terminology and the processing rules were partially contradictory
>> from the beginning -- and that apparently was caused by historical
>> circumstances and pressure exerted on the "TCP committee" over a
>> long time span in these old days.
>
> My take is that the "corrections" would not have interoperated
> with the deployed code, and that even then, there was no ROI
> in e.g. changing the semantics of the urgent pointer.
>
>
>> Therefore, returning to the picture painted above, the *first*
>> step should be to formulate strong recommendations for how
>> applications and in particular implementations of applications
>> should make use of TCP Urgent Processing (if any).
>> This IMO should be based on the concepts explained in the two
>> definitions from RFC 793 quoted above.
> [....]
>> Since decades, Telnet makes use of URG, and apparently this
>> all works (almost) properly, independent of the underlying
>> TCP stacks' behavior at the sending and receiving side.
>
> Well, the "underlying TCP stacks' behavior" is homogeneous -- everyone
> implements the same non-compliant behavior.
>
>
>
>> (BTW: the discussion in RFC 1123 seems also to be disturbed.)
>> The reason seems to be simple: being defensive, and not
>> assigning semantics to the data delivered with the sending
>> side's urgent call.
>> According to RFC 854, the Telnet client sends a DM NVT control
>> character which is a transparent NOP for the Telnet server.
>> Thus, it does not matter for the Telnet server how the TCP
>> stack interprets the Urgent pointer.
>
> It should matter. If a system sends data with the deployed semantics for
> the UP (i.e., it points to the byte following the last byte of urgent
> data), but the receiver implements the compliant (RFC 1122) behavior
> (i.e., points to the last byte of urgent data), then one control byte
> (after DM NVT) will be skipped.
>
> OTOH, if a system sends data with the IETF-compliant (RFC1122)
> semantics, and the receiver implements the deployed semantics, the
> "urgent byte" will not be skipped (this would be safe for telnet,
> as you point out).

Appparently, we all here also suffer from the ambiguity of the term
"urgent data".  It looks like RFC 1122 calls "urgent data" what
RFC 793 tells "the data octet associated with the URGENT [SEND] call",
whereas RFC 793 associates the term "urgent data" with the data declared
obsolete and hence (cumulatively) to be skipped over in urgent mode,
i.e. the data already in the transmisison pipeline when such URGENT
SEND call is made.

"A sequence of urgent data of any length" in RFC 1123 does not
refer to an URGENT SEND with more than 1 data octet, but to the
case of multiple URGENT calls that cumulatively add to the urgent
data (in the sense of RFC 793).  RFC 793 talks about "_the_ octet"
in the URGENT call.  RFC 793 explains the URGENT flag for the SEND
call as the means to send an "urgent indication", and on page 42,
it says:
    To send an urgent indication, the user must also send at least
    one data octet.
Indeed, one might argue that the whole problem stems from the fact
that TCP did not assign the urgent indication, i.e. the cutting point
separating the data to be processed in urgent mode (skipped over,
ignored) from the data where normal mode resumes.  Since the TCP SN
is attached to octets in the data stream and not to the 'separation
line' between two octets, the URGENT call needs to send data --
otherwise the sending TCP would not have an opportunity to transport
the urgent indication to the receiver.



>> (1) Give recommendations for application protocol designers
>>     and application programmers of how to best use the common
>>     Sockets API for Urgent Processing, in order to accommodate the
>>     legacy problems and the inconsistencies in the specifications.
>>
>>     This should target BCP status, and it should have a tight
>>     milestone of not more than 6 months for passing to the IESG.
>>
>>     Most elements of such document already are in
>>     draft-gont-tspm-urgent-data-01 ;
>>     my personal expectations for the basic recommendations are:
>>
>>     o  MUST always use OOBINLINE;
>
> Agreed.
>
>
>>     o  SHOULD only send a single octet in the urgent call;
>
> This would contradict RFC 793 and RFC 1122 (?)

No, not strictly.  :-)

When the URGENT SEND is issued with one octet of data and with PUSH,
the sending TCP would have to send a segment with one octet that
the receiver should deliver to the application in normal mode.
We provisionally agree that RFC 793 on page 56 intends to perform
SND.UP <- SND.NXT-1  _after_ processing this octet, the UP would
point to this single octet, thus designating all previous data as
urgent (definition of URG on pg. 84: "... urgent processing as long
as there is data to be consumed with sequence numbers less than the
value indicated in the urgent pointer").
RFC 1122 -- under the influence of the many implementers that
erroneously assumed OOB data in TCP and re-interpreted the (normal
mode!) data sent in the URGENT call as "Urgent Data" -- now talks
about "a sequence of urgent data" for the case a user hits "Interrupt"
multiple times, but AFAICS still assumes as in RFC 793 a single octet
sent in each URGENT call, and replaces the definition of URG in RFC 793
with the apparent re-definition, "the urgent pointer points to the
sequence number of the LAST octet in a sequence of urgent data."
But this is exactly what RFC 793 had specified, yet using another
definition of "urgent data".  That use of the same term with different
meaning was, and still is, the basic reason for the trouble we see.

Hence, as long as each URGENT SEND call carries exactly one octet
of data, RFC 1123 and (most parts of) RFC 793 state exactly the
same rules, but with effectively different terms.
The ambiguity only arises when an URGENT SEND carries more than one
octet of data and this octet string is considered the "urgent data",
contrary to the basic intent of RFC 793.


>>     o  SHOULD use syntactically transparent data in the urgent call
>>        (equivalent to DM == NOP in Telnet);
>
> But... what about the deployed sender to RFC112-compliant receiver
> case sketched above? Or are you also thinking about updating the
> semantics of the UP so that they reflect the deployed code?

I do not think that we should update the semantics of the UP.
IMO, we should take the definition of URG on page 84 of RFC 793
as the "constitutional law".
The real need to change is fixing one single definition of
"urgent data", if we want to remain stick with that fuzzy term.
Once this is done, it will become clear without any controversy
which parts of the various RFCs need to be updated to make use
of that single definition, and change the wording where necessary.

My personal preference (but once more: I will not insist on it)
would be to avoid that legacy term already meaning different things
in the brains of folks, as far as possible; each use of it incurs
the risk to again raise the erroneous expectation of something like
"OOB data" in TCP.

One possibility would be to denote the data with SN < UP as the
"OBE data" ("outdated by events") (as soon as an urgent signal
has been sent out), and denote "the octet associated with the
URGENT call" (RFC 793) as the "restart of Normal Mode data".



>>     o  intermediate systems MUST NOT scrub the TCP URG flag
>>        (URG is an entirely end-to-end indication);
>
> Agreed. But... they have been deployed. And Cisco PIX is widely
> deployed, and does scrub the URG bit!

There should be a clear position that this behavior is in violation
of the TCP standard.
Further, it breaks all integrity protection security protocols.
Only real B2BUAs (Proxies, NAT ALGs, etc.) that are able to share
security associations with both ends of the TCP connection they
effectively operate on as a man-in-the-middle, are allowed to
interfere with a fully end-to-end protocol like TCP.


>>     o  a hint that a parallel TCP connection ('control channel')
>>        SHOULD be used if an application needs true OOB data transfer.
>
> Agreed.
>
>
>
>> Based on the captured intent of the original specifications,
>> therefore, a unique definition for "Urgent Data" should be found.
>> In RFC 793, three possible candidate definitions are sublimely used:
>>
>>     a) the data 'temporally' ahead of the Urgent Pointer,
>
> Did you mean "behind"?

Hmmm.  Why?
With 'temporally ahead' I mean the data sent earlier (with smaller
UTC values) and still in the pipeline (send buffer, network, and
unread receive buffers), when an URGENT call is made (and thus having
a numerically smaller SN), and which thus shall be skipped over by
the TCP receiver in urgent mode, per RFC 793.
The urgent pointer terminates the data to be processed in urgent mode.
RFC 793 defines "... urgent processing ... [for] data ... with
sequence numbers less than the value indicated in the urgent pointer."


>>        i.e. the all the buffered data that are being invalidated
>>        by the 'urgent' call (TCP 'SEND' call in the abstract TCP
>>        'User Interface' of RFC 793, section 3.8, pp. 46/47, with
>>        URGENT flag set);
>>
>>     b) the data sent with the 'urgent' call;
>>
>>     c) the union of both a) and b).
>
> Well, ironically, what's "urgent" is the first non-urgent data! :-)

Hmm -- that kind of reasoning is stuck with the OOB data paradigm!
The TCP design does not have such data; TCP carries an indication
to enter urgent processing, i.e. to skip over and ignore some data;
since there's only a single data stream, *skipping* is urgent!

> i.e., with "urgent data" being whatever you read while being in "urgent
> mode", you basically want to skip those data, such that you can quickly
> get to the first non-urgent data (i.e., get out of urgent mode).

Exactly!


>> Note #3: The ambiguity in RFC 793 mostly stems from that it remains
>>   underspecified whether the urgent pointer refers to the state of
>>   the sequence number at entry to the event processing or at its
>>   end, i.e. whether (on pg. 56)  "SND.UP <- SND.NXT - 1" should
>>   happen before of after processing the data handed over in the
>>   SEND with URGENT flag set.
>
> It should happen *after*. Remember that a TCP is allowed to send
> sequences of urgent data of any length. (yeah, I hate to use the term
> "urgent data"... but you know what I mean).

Sigh, no, some parts of your draft have confused me even more,
by using that term in different meanings as well!

So, the more I consider the topic, the more my desire increases
to indeed get entirely rid of that term.
Otherwise, notwithstanding a clear definition, people will come
back again and again and associate it with OOB data.


> [...]
>
>> Thus, the second delivery of the TCP URG work item should be:
>>
>> (2)  A Normative (Standards Track) document clarifying the
>>      definition of the term "Urgent Data" for TCP and Updating
>>      primarily RFC 793 and RFC 1122, but for consistency also
>>      RFC 854 and RFC 1123, to make them conformant to the
>>      clarified definition and provide consistent and self-
>>      consistent processing rules.
>
> What, specifically, would this I-D update?

That depends on the definition(s) the WG comes to consensus to use.
I have shed some light on how I would tackle the issue, but I will
support any reasonable way to achieve full consistency in the
definitions and their use in the (updated) specs, as long as it
remains clear that we do _not_ change the original TCP philosophy
and try to introduce actual OOB data.



>> I'd offer to work as a co-editor with Fernando Gont for a TCPM
>> work item shaped along the ideas and guidelines presented above,
>> but more volunteers to help with completing the investigations
>> of the TCP-related IEN documents would be heartfully welcome,
>> in order to give the whole effort the best possible foundation
>> to be in the spirit of the original designers of TCP.
>
> So you argue that tcpm should adopt draft-gont-tcpm-urgent-data,
> and use that I-D as a basis for the BCP document, and also work
> on another (std track) I-D that we'd both co-author?

Some parts of the current draft might go into the second document.
To reduce the risk of controversies and lenghty debates, IMO the BCP
ideally should not talk about internal details of the TCP stack at all.
Of course, the BCP preferably should not use terminology not
unambiguously defined.


> BTW, is there a need to actually dig at the IENs, etc.? I'm not saying
> it wouldn't be worth the effort, but... I' not sure it would be actually
> "required"/needed to clarify TCP's urgent mode, etc.

If there is a sound support in the WG to base the work on the
definitions in the Glossary of RFC 793 without further confirmation
of the original rationale applied in the development of TCP,
I'll be fine with skipping that tiresome work!

However, I am well aware of a strong position in the WG to not change
the original intent of the TCP specs in any way; and thus, proofing
more evidence that the intended work is strictly in line with that
philosophy might help to speed up the process, and to gain strong
consensus on it.


> Thanks!
>
> Kind regards,
> --
> Fernando Gont
> e-mail: [e-mail] || [e-mail]
> PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1


Kind regards,
  Alfred Hönes.

-- 

+------------------------+--------------------------------------------+
| TR-Sys Alfred Hoenes   |  Alfred Hoenes   Dipl.-Math., Dipl.-Phys.  |
| Gerlinger Strasse 12   |  Phone: [phone], Fax: -18         |
| D-71254  Ditzingen     |  E-Mail:  [e-mail]                     |
+------------------------+--------------------------------------------+