Re: [rmcat] rmcat Digest, Vol 21, Issue 2

[Keith Winstein <[e-mail]>]

Our experience with packet reordering has actually been quite
different. We (Mosh) have found that packet reordering can be
extremely common on 802.11n networks when the "block acks" feature is
used and a link is saturated, at least with some receiving 802.11n
chipsets, including the ones in MacBooks.

In Mosh version 1.2.2, we turned on an advisory warning when packets
were reordered in the network, because we thought that in practice
reordering was rare enough to be notable and might signify an
attempted attack.

After the release, we got a lot of complaints that the warning was
showing up frequently for users on enterprise 802.11n networks using
the MacBook and a few other devices. On a full-throttle TCP download
to a MacBook Pro, we found that something like 30-50% of all TCP segments
were being delivered out of order! And Mosh's UDP datagrams were seeing
similar treatment when there was a back-to-back flight of them. We ended up
disabling the warning in v1.2.3.

As I understand, the block acks feature means that a sender may
transmit a whole chunk of frames (some of which will be received
without error and some won't), and then the receiving station can tell
the sender which frames were successfully received and which need to
be retransmitted. In the meantime, the receiver can deliver the
successfully-received datagrams to IP or wait.

RFC 3366 3.1 essentially advises link designers that they should use a
receive-side reordering buffer to try to put datagrams back in order
before delivery to IP, but our testing suggests that common 802.11n
receive chipsets don't do this in practice.

So our experience has been that in the contemporary Internet (at least
on some 802.11n networks), reordering can be much more common than we
had initially believed.

On Fri, May 2, 2014 at 3:00 PM,  <[e-mail]> wrote:
> ---------- Forwarded message ----------
> From: Randell Jesup <[e-mail]>
> To: [e-mail]
> Cc:
> Date: Fri, 02 May 2014 05:49:16 -0400
> Subject: Re: [rmcat] draft-sarker-rmcat-eval-test-00, test case 4.4 RMCAT Flow competing with a long TCP Flow
> On 4/30/2014 1:51 PM, Colin Perkins wrote:
>>
>> On 30 Apr 2014, at 07:50, Zaheduzzaman Sarker <[e-mail]> wrote:
>>>
>>> Are you saying re-ordering is not a phenomena that we can observe in the current Internet? And we should not have test cases to handle re-ordering of packets in RMCAT?
>>
>> One of my students collected some UDP streaming measurements from our university to hosts on residential networks, covering all the main UK ISPs, a couple of years ago. With everything using wired connections and UDP we saw a couple of hundred packets delivered out of order, from ~250 million packets sent. I can dig into the details of the re-ordering patterns if there’s interest. We didn’t analyse them too closely at the time because packet loss and delay spikes were much more common.
>
>
> Having it not fall over on (some) reordering: good.  My experience has been only specific unusual network/hardware situations cause more than 0.0x% reordering of packet delivery (at the IP level, ignoring things like TCP retransmit, which is not an IP-level reordering in the way I'm discussing).
>
> I've seen significant reordering only on a network (an old office network of mine!) where we had multiple bonded T1's and the packets went across one of them randomly (I presume).  I think I got to low-single-digit percent or more likely 0.n% (like 0.1-0.9%) reordered.  There may also be some wireless networks where this is more likely than on normal wired; I haven't noticed it on WiFi though.
>
> It's easy to accidentally set up DummyNet/etc to cause massive packet reordering, however.  I do not think that should be a primary or even secondary testcase, and (to be honest) I don't think we should care much about degenerate reordering cases.  I'm not even sure we should bother spending much energy in our evaluation tests making it happen at all (which isn't to say that the algorithm authors shouldn't consider it and if they feel it would affect them, test it or discuss it - plus they will see a small amount of it on real networks).
>