Re: Review draft-ietf-tsvwg-byte-pkt-congest-04 ?

[Bob Briscoe <[e-mail]>]

Fred,

Tx for review. inline responses...

At 00:35 31/07/2011, Fred Baker wrote:

>On Jul 28, 2011, at 11:14 PM, Bob Briscoe wrote:
> > Would you be willing to review this, now it is well-baked?
> >
> > It had wide review earlier in its life, but only 2 or 3 reviews 
> since we reorganised it to make it easier to read (hopefully).
>
>Sure Bob.
>
> > Abstract
> >
> >    This memo concerns dropping or marking packets using active queue
> >    management (AQM) such as random early detection (RED) or pre-
> >    congestion notification (PCN).
>
>This sentence seems to mix apples and orangutans. RED is an AQM 
>procedure, meaning that it decides by some mechanism what packets to 
>do something to with a view to active queue management. The 
>structure of the sentence would suggest that PCN is also an AQM 
>procedure. I'll argue that either the AQM procedure uses dropping 
>and/or PCN for signaling or the drop/mark mechanism uses AQM to 
>decide what do affect and the signaling procedure to comminicate. 
>While they both (RED/AQM and PCN, as well as dropping) merit mention 
>in the abstract, I wouldn't put them parallel in the sentence. You 
>can't really say "I think I'll turn off RED and turn on PCN".

The structure of your sentences above implies you are considering PCN 
to be the name of the encoding of the signals, or perhaps the field 
they are encoded in. That's not so.

PCN is an AQM algorithm [RFC5670] that uses an encoding of the DS & 
ECN fields to signal [RFC5696]. They aren't PCN signals, they are 
signals in the DS/ECN fields from a PCN algorithm.

(Confusingly, PCN is also the term for a whole admission control 
architecture [RFC5559]).

 From RFC5670:
"
    Note that although [<http://tools.ietf.org/html/rfc3168>RFC3168] 
defines a broadly RED-like (Random Early
    Detection) default congestion marking behaviour, it allows
    alternatives to be defined; this document defines such an
    alternative.
"

Turning off RED and turning on PCN is precisely what you do to enable 
PCN (for a particular set of DSCPs).

If you're happy with this explanation, is the abstract still OK as it stands?

> >    We give three strong recommendations:
> >    (1) packet size should be taken into account when transports read
> >    congestion indications, (2) packet size should not be taken into
> >    account when network equipment creates congestion signals (marking,
> >    dropping), and therefore (3) the byte-mode packet drop variant of the
> >    RED AQM algorithm that drops fewer small packets should not be used.
>
>Wow. Before I go into the draft itself, let me comment on how I have 
>historically come down on Van and Sally's debate here.
>
>If one is dropping traffic to signal congestion, the only loss that 
>TCP or SCTP is cognizant of in that sense is the loss of data 
>packets. Dropping a SYN, SYN-ACK, INIT, or whatever prevents a 
>session from opening, dropping a FIN/RST/etc prevents it from 
>closing, and dropping an ACK delays a signal releasing state in the 
>transport and potentially triggering new transmissions. None of 
>those have anything to do with the congestion window. Hence, if I 
>could understand TCP traffic in the place that queues are managed 
>(which I often cannot - the data is encrypted, or I'm on another 
>card that simply doesn't have that visibility) I would restrict 
>congestion-management signals in the form of loss to data segments.
>
>Ditto if someone is marking TCP/SCTP traffic; ECN depends on a 
>signal returning on data packets, and as a result is primarily 
>implemented (by TCP) in data packets. One could argue that I could 
>learn CE from an Ack or whatever, but I'm not sure why I would 
>bother, as communicating it back requires saving it somewhere as 
>opposed to simply including it in a reply I'm going to send anyway.
>
>The question of doing it on bytes vs packets was never, to my 
>knowledge, about preventing the loss of small packets. It was about 
>working effectively with TCP's congestion control mechanisms and 
>managing router resources. Van wanted us to integrate packet depth 
>because router resources were measured in buffers, which are the 
>same size regardless of how much data they contain (in some 
>implementations), and Sally wanted it on bytes because that was a 
>reasonable heuristic for identifying data segments, which interact 
>better with TCP's mechanisms.
>
>Having for 30+ years worked on equipment that was able to chain 
>relatively small buffers to store a larger message, Van's argument 
>has always seemed incomplete to me, and Sally's makes sense 
>mechanistically. So I have generally come down on the side of bytes...

Yup. I'm aware of this debate. The doc addresses it head on.


>But, whatever. On to your document...
>
>I have dropped your entire document into my mail window. Text I 
>don't have a problem with I will simply hack out, copying back text 
>I have questions about. I assume that you can find it in the 
>document by the markers I leave behind or by common search 
>functions. I'm not going to say "[snip]"...
>
> > 1.  Introduction
>
> >    When notifying congestion, the problem of how (and whether) to take
>                    ^...the participants in a session of the onset of...

OK


> >    packet sizes into account has exercised the minds of researchers and
> >    practitioners for as long as active queue management (AQM) has been
> >    discussed.  Indeed, one reason AQM was originally introduced was to
> >    reduce the lock-out effects that small packets can have on large
> >    packets in drop-tail queues.
>
>True, but I'll note that large packets can have a lock-out effect on 
>small packets as well. RFCs 896 and 970, at least, worried about 
>small packets not because there were so many of them, but because 
>they implied the presence of additional IP and TCP headers and as a 
>result relatively inefficient use of the line.

Yup. Just to check: you're not asking us to change anything in this 
para, just noting?
(A transport benefits from using the Nagle algo without the network 
additionally biasing drop probability towards small packets.)


> >    The byte vs. packet dilemma arises at three stages in the congestion
> >    notification process:
> >
> >    Measuring congestion:  When the congested resource decides locally to
> >       measure how congested it is, should the queue measure its length
> >       in bytes or packets?
> >
> >    Encoding congestion notification into the wire protocol:  When the
> >       congested network resource decides whether to notify the level of
> >       congestion by dropping or marking a particular packet, should its
> >       decision depend on the byte-size of the particular packet being
> >       dropped or marked?
> >
> >    Decoding congestion notification from the wire protocol:  When the
> >       transport interprets the notification in order to decide how much
> >       to respond to congestion, should it take into account the byte-
> >       size of each missing or marked packet?
> >
> >    Consensus has emerged over the years concerning the first stage:
> >    whether queues are measured in bytes or packets, termed byte-mode
> >    queue measurement or packet-mode queue measurement.  This memo
> >    records this consensus in the RFC Series.  In summary the choice
> >    solely depends on whether the resource is congested by bytes or
> >    packets.
>
>If you're recording a consensus for posterity, I think it needs to 
>be more clearly stated. Do you mean "the consensus is that both 
>choices are viable; the implementor or operator should choose to 
>measure whatever is congesting his links"? I personally don't hear a 
>clear statement here about (1).

All the clear statements are in S.2 Recommendations. So we need to say:

"Section 2.1 of this memo records this consensus in the RFC Series."


> >    The controversy is mainly around the last two stages: whether to
> >    allow for the size of the specific packet notifying congestion i)
> >    when the network encodes or ii) when the transport decodes the
> >    congestion notification.
>...
> >    The primary purpose of this memo is to build a definitive consensus
> >    against deliberate preferential treatment for small packets in AQM
> >    algorithms and to record this advice within the RFC series.  It
> >    recommends that (1) packet size should be taken into account when
> >    transports read congestion indications, (2) not when network
> >    equipment writes them.
>
>
>I have read the rest of the paper, but let me stop here and make 
>some generalizations.
>
>First, I expected to find some data or pointers to data. There are 
>in fact some pointers to other documents. The vast majority of the 
>statements in the document that might be supported by data appear to 
>say "80% of those we might have expected to respond didn't, but..." 
>or something like that. I'd be a lot happier with statements of the 
>form "we ran a test..." or "we were able to construct a proof that 
>the mathematics tells us..."

The subsections of S.3 give all the motivating arguments.
* Some are in the form of a proof (S.3.1 on scaling).
* Some (S.3.2) are in form of layering arguments about where the more 
generic functions should be placed. Are we going to make AQM 
decisions depend on inspection of the structure of transport 
protocols like TCP, SCTP, DCCP, RTP etc etc. (assuming no IPsec)?
* Yes, some (e.g. S.3.3 on DoS) ought to reference empirical data to 
back them up. We need to add a ref here to tests of the performance 
for small packets of byte-mode drop (such results were not intended 
to show that small packet DoS works, but they do).
* Some are in the form of statements of a security vulnerability 
(S.3.4 small != control)

If performance were the issue, then I would agree that references to 
performance studies would be necessary. However, whether a future 
protocol design can be made to go slightly faster if the network does 
byte-mode drop, versus the end-system taking account of packet size 
is a moot question in the presence of larger arguments of scaling and 
security vulnerabilities.

This document isn't about which AQM design gives the best performance 
if we hold the TCP design static. It's about the constraints we are 
faced with for future protocol design beyond TCP.


>Second, I disagree with the argument about a "perverse favoring of 
>small packets". At least in implementations I'm familiar with, the 
>policy isn't "drop only large packets"; it's "make the drop 
>probability of a packet proportional to the ratio of its size to the 
>bit rate of the link".

Yup.

>If all packets are 9000 bytes and you're on an STM-4 link, the drop 
>probability of a packet is proportional to (8*9000)/622000000. If 
>all packets are 64 bytes, the drop probability of any given packet 
>is proportional to (8*64)/622000000, which is smaller, but there are 
>9000/64 times as many to choose from. Any given bit on the wire has 
>the same probability of being contained in a packet that got lost.

I believe there's a mistake here. To see why it's a mistake, I'll 
carry through the numerical example to the end (as in Appx A.2).
Assume the constant of proportionality in both your cases is 100, that is:
  Drop prob of large pkts is 100*(8*9000)/622000000 ~ 1.2%
  Drop prob of small pkts is 100*(8*64)/622000000   ~ 0.0082%

Let's compare the losses from a stream of 64,000 large pkts vs 
9,000,000 small pkts
Each stream contains the same number of bytes (64,000*9,000 = 
9,000,000*64 = 576MB) so it will last for the same time on the same link.

Number of large pkts dropped by byte-mode drop
  = 100*(8*9000)/622000000 * 64,000
  = 1.2% * 64,000
  = 741 large packets

Number of small pkts dropped by byte-mode drop
  = 100*(8*64)/622000000 * 9,000,000
  = 0.0082% * 9,000,000
  = 741 small packets

Number of Bytes dropped in large packets by byte-mode drop
  = 741*9,000B
  = 6.67MB
Number of Bytes dropped in small packets by byte-mode drop
  = 741*64B
  = 47.4kB

With byte-mode drop, any given bit on the wire does NOT have the same 
probability of being contained in a packet that got lost.

Yes, there are 9000/64 times as many small packets to choose from. 
Therefore in order to drop a bit with the same probability no matter 
what size packet it is in, each small packet should be dropped with 
the SAME probability as a large packet (packet-mode drop).

With the same probability per packet, the dice is thrown 9000/64 
times more often for small packets, but if the outcome is a drop, the 
amount of bytes lost is only 64/9000 times as much. These two already 
cancel each other out. So there's no need to deflate the probability 
of dropping small packets further.

>On the other hand, if packets are variable in size and you're keying 
>off packet depth, a bit on the wire that is contained in a small 
>packet has a much greater proportional probability of being tagged 
>than one in a large packet.

I think your phrase "keying off packet depth" means measuring queue 
length in packets, rather than deciding the probability of dropping a 
particular packet based on its size. If so, I don't believe the 
second part of the sentence is correct.

When a small packet arrives at a queue of packets of different sizes, 
its /own/ size doesn't make any difference to the number of packets 
in the queue. That's why the draft separates out how you measure the 
queue length, from how you calculate the probability of tagging a 
particular packet.

>I don't see that one is incented to send small packets; if the 
>probability that a given bit gets lost is the same regardless of 
>size, it seems to me that it takes size out of the discussion.

Exactly. I agree that the goal is for the probability of losing a bit 
to be independent of the size of packet it happens to be in.

But byte-mode drop fails that test (miserably). Packet-mode drop passes.


>I'll go back and read the paper again a couple of times before 
>commenting further.

Thanks again for your time on this.



Bob


________________________________________________________________
Bob Briscoe,                                BT Innovate & Design