Re: [vwrap] Technical basis for VW client in a web browser?

[Dahlia Trimble <[e-mail]>]

Morgaine, thanks for providing some in-depth insight into this issue.

Often the arguments I hear against UDP are based around duplicating TCP
functionality at the application level. I do not believe this to be an
adequate design goal, and if the functionality that TCP provides is deemed
necessary then it would seem a poor choice to use any other protocol.

One problem I've found with TCP is that it's used as a delivery mechanism
for a serial stream of data. One end point effectively pushes data into a
serial queue and the other end pulls it off the queue. If there is
congestion, the queue cannot continue to deliver data to the application
until the missing pieces are resolved. Multiplexing several message streams
into one TCP stream will effectively shut down *all* streams while there are
any missing packets for which resends must be negotiated. The application
has no means of overriding this. Hence, in an implementation where
time-critical messages are sharing a serial message queue with other less
critical messages, they must wait their turn while all preceeding message
are successfully received by the other end point. UDP, by contrast, is not
subject to these effects as any message can be pushed out the interface at
any time regardless of what other queues may exist. Cleverly designed
protocols may use optimized acknowledgement schemes such as the "appended
ack" scheme in the LLUDP (Linden Lab UDP) protocol, or by more sophisticated
means if desired. Simply hammering out messages at full speed until the
application gets back the desired response would *not* be what I consider a
practical scheme, due to many of the reasons you described about "MANAGING
AGGRESSIVE FLOWS". It's also likely that under heavy network congestion
periods, a well-designed UDP based message protocol might actually reduce
traffic as some messages which lose their value as time passes will not need
to be resent in the case of packet loss, TCP streams however *must* resolve
all lost information to maintain a serial stream and application end points
cannot discard any messages once they have been placed into the stream.

I agree that poorly designed UDP base protocols are not desirable, but I
would rather not see UDP be regarded as a poor choice simply because poor
implementations exist. I'm sure I could find equally poor implementations
using TCP if I tried, but I would not want to see TCP be brushed off
either.  It would seem a useful design goal that a protocol could
selectively route messages thru the best available mechanisms at run time
for each message type. and fall back to less optimal choices if the best are
not available.


On Mon, Dec 20, 2010 at 8:44 AM, Morgaine <[e-mail]>wrote;wrote:

> On Mon, Dec 20, 2010 at 12:56 AM, Dahlia Trimble <[e-mail]>wrote;wrote:
>
>>
>> If anyone has any evidence of internet pathways that selectively favor TCP
>> over other traffic, I'd be interested in seeing it.
>>
>
>
> I have first-hand knowledge of this.  I worked for several years at the
> Network Operations Centre of a top-tier ISP, and one of my duties was
> looking after service routers and firewalls.  Policy-based routing and
> firewall access lists are configured with rulesets which are processed
> sequentially and eat up router CPU, which is a finite resource.
>
> Under off-peak conditions, packet loss is quite rare in the absence of
> interface or line faults, and router CPUs are scaled to handle the expected
> load so packets are never dropped willfully.  When networks are congested
> however, which unfortunately is not uncommon during peak hours owing to the
> common practice of oversubscribing capacity (or poor scalability planning),
> CPU load often reaches critical levels, and routers are configured to
> prioritize certain payload types in favor of others when this happens.
>
> TCP always gets top priority because it carries HTTP which is most closely
> tied to business revenue.  In contrast, UDP (and also ICMP Echo) are
> normally configured right down the bottom end of the priority list, so under
> peak load when the CPU has to make a choice what to drop to stay within safe
> operating limits, UDP gets the chop first.  This was business as usual at
> the ISP, and that's how the network designers wanted the traffic priorities
> configured.  (I was merely implementing policy, not creating it.)
>
> Gaming fans sometimes complain that ISPs are reducing the quality of
> service of their UDP traffic, and in some sense it's true.  From my
> experience it's not done maliciously nor as an conscious policy of network
> non-neutrality, but simply as a means of protecting the more prized resource
> of TCP payloads when operating conditions mandate that something has to be
> thrown away.
>
> On the positive side, I've never known UDP packets (nor any other kind) to
> be dropped willfully for the above reason when all equipment is working
> within safe design limits and there is enough capacity to carry them.  If it
> happens off-peak then something is very wrong with network sizing, or the
> equipment is faulty.
>
> Unfortunately, that's not the end of the saga with UDP.  It's just the
> beginning, because there is another big reason for dropped packets, and this
> one occurs even when equipment is working within its designed operating
> limits:  traffic shaping.
>
> IP traffic shaping is performed by packet queuing as a first resort, to
> slow down traffic in the hope that the source notices and adapts.  If the
> packet rate doesn't slow down then the method of last resort is to drop
> excess packets when queue buffers hit their configured highwater marks.  TCP
> implements a lot of things to mitigate packet loss, such as slow start,
> exponential backoff and transmit pacing, with the purpose of adapting
> transmit rate to receipt rate across paths of limited bandwidth so that
> traffic-shaping routers don't enter their drop state.
>
> UDP has no such flow control, so the onus falls upon the UDP application
> endpoints to carry out adaptive flow control themselves.  The likelihood of
> this being done by UDP applications as effectively as it is done in today's
> finely honed TCP stacks is very low.  It may not even be done at all.
>
> As a result, UDP packets can get dropped for being bad network citizens and
> not slowing down in response to packet queueing and transmit pacing.  UDP
> applications may think that they're free of the shackles of TCP flow
> control, but they're not.  They either slow down when given the hint, or
> their traffic gets the chop.  As a result, a UDP application that is
> oblivious of end-to-end timing should expect packet loss when the network
> acts to protect itself against congestion.  See RFC-2309 for more details
> about this issue, in particular "MANAGING AGGRESSIVE FLOWS".  There is no
> free lunch for UDP packets.
>
> This is why the best advice to give prospective users of UDP is "*Don't,
> unless your application is tolerant of packet loss, packet duplication,
> packet delay, corrupted packets, and out of order delivery.*"  To try to
> work around these properties of UDP and make it reliable while
> simultaneously not impacting on the congestion controls of TCP over the
> shared path is highly unlikely to be successful, unless you reimplement the
> clever control features of TCP, and do it compatibly.  Bulldozing your UDP
> packets through a shared network is not a solution, and will fail.
>
> *[PS. The problems don't even stop there, as there are further causes of
> UDP packet loss.  One is the effect of TCP flow-control synchronization on
> UDP loss rate over congested paths, which counter-intuitively negates any
> benefit that could result from reducing UDP traffic rates because TCP
> synchronization picks up any bandwidth slack that reducing UDP traffic has
> freed.  As a result, TCP congestion actually increases UDP packet drop
> rates.  (This has been a topic of research.)  Networks protocols have very
> complex behaviors.]*
>
>
> Morgaine.
>
>
>
>
> ======================================
>
>
> On Mon, Dec 20, 2010 at 12:56 AM, Dahlia Trimble <[e-mail]>wrote;wrote:
>
>> I have used both TCP and UDP in VW applications. I've found that TCP has
>> acceptable latency and is not really any worse than UDP when either are
>> tried over a clean, highly functional connection. I've not seen any routers
>> which drop UDP packets in favor of TCP, and I've not seen any evidence of
>> better quality TCP connections than UDP in any of my tests. To the contrary,
>> I've seen UDP perform much better when network conditions are less than
>> optimal as small messages can be sent immediately and repeated as needed
>> without waiting for prior message acknowledgement or waiting for a TCP
>> stream to recover in the event of dropped packets.
>>
>> TCP seems to be favorable when latency is not critical as it's generally
>> (but not always) easier to use. UDP seems favorable when latency is critical
>> as it allows the programmer to control network traffic and tailor it to the
>> application requirements.
>>
>> If anyone has any evidence of internet pathways that selectively favor TCP
>> over other traffic, I'd be interested in seeing it.
>>
>>
>> On Sun, Dec 19, 2010 at 3:26 PM, Meadhbh Hamrick <[e-mail]>wrote;wrote:
>>
>>> do we really know that UDP is what we want, even for low latency? if
>>> you're multiplexing messages over a websocket connection, it's highly likely
>>> it'll be an existing connection (i.e.- it's likely one tcp/ip connection
>>> will carry several multiplexed websockets messages.)
>>>
>>> in my tests, UDP doesn't do much better than TCP if you're near the
>>> network rate as it seems a lot of routers tend to dump UDP packets first.
>>>
>>> most modern OSes now have api calls to let you disable TCP slow-start.
>>>
>>> i guess what i'm saying is it might be a good idea to define messages in
>>> a way so they're transport agnostic. that and I would wager that any latency
>>> improvements from UDP are dwarfed by latency introduced by application layer
>>> mechanisms to replace TCP's flow control & resend semantics.
>>>
>>> just my $0.02.
>>> On Dec 19, 2010 2:34 PM, "SM" <[e-mail]> wrote:
>>>
>>> _______________________________________________
>>> vwrap mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/vwrap
>>>
>>>
>>
>> _______________________________________________
>> vwrap mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/vwrap
>>
>>
>
> _______________________________________________
> vwrap mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/vwrap
>
>