Re: Comment on draft-ietf-tsvwg-byte-pkt-congest-05.txt

[Bob Briscoe <[e-mail]>]

Joe & Fred,

At 09:49 14/11/2011, Joe Touch wrote:
>Hi, Fred,
>
>On 11/13/2011 11:52 PM, Fred Baker wrote:
>...

(I've added the this context, given I've taken so long to reply - sorry):
 >    If the resource is bit-congestible, the implementation SHOULD measure
 >    the length of the queue in bytes.  If the resource is packet-
 >    congestible, the implementation SHOULD measure the length of the
 >    queue in packets.

>>I think the statement is fine as far as it goes, but it has two
>>remaining issues. One is the question of when an interface is
>>bit/byte congestible and when it is packet-congestible; I'm sure that
>>means something specific to the authors, but will be far less clear
>>to an implementer - every interface has a rate, and few if any
>>measure their rates in packets per unit time, so specifically what is
>>in view as a "packet-congestible interface"?
>
>Some examples in the text might be useful.


We did give examples at the point where we define 
'packet-congestible' earlier (pasted below).

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1.1. Terminology and Scoping

    Bit-congestible vs. Packet-congestible:

       Examples of packet-congestible resources are route look-up engines
       and firewalls, because load depends on how many packet headers
       they have to process.  Examples of bit-congestible resources are
       transmission links, radio power and most buffer memory, because
       the load depends on how many bits they have to transmit or store.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

>Bit/byte congestion:        - storage that assigns space based on 
>the packet size
>         - interfaces whose output bandwidth is overloaded
>         - processing that is length-dependent (e.g., full packet
>         hashes or encryption)
>
>Packet congestion:
>         - storage that assigns space on a per-packet basis,
>         i.e., that has fixed slots (for max packets)
>         - interfaces whose output slots are overloaded
>         (with fixed slots per packet)
>         - processing that is per-packet dependent (e.g.,
>         header processing, header hashing, tunneling)
>
>It is useful to note that there are some variants of these, e.g., 
>where there are a small number of different buffer sizes (coarsely 
>per-bit/byte), or ones with combined bottlenecks (there can be 
>multiple places where resources are under contention!)

[BB]: To help implementers understand how to interpret the guidance 
in a more complex cases like this we talk about both cases as 
specific examples in the text. Search for 'buffer carving' and 
'hybrid forwarding system' respectively.


>It's useful to explain what to do when there are combined 
>bottlenecks in both places (alleviating only one might not suffice). 
>I wonder what the algorithm should be in that case...

[BB]: To avoid making the draft too complicated, we fudged a bit by 
saying the following:

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
    If we now imagine a hybrid forwarding system with transmission delay
    largely dependent on the byte-size of packets but buffers of one MTU
    per packet, it should strictly require a more complex algorithm to
    determine the probability of congestion.  It should be treated as two
    resources in sequence, where the sum of the byte-sizes of the packets
    within each packet buffer models congestion of the line while the
    length of the queue in packets models congestion of the queue.  Then
    the probability of congesting the forwarding buffer would be a
    conditional probability--conditional on the previously calculated
    probability of congesting the line.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

Given Fred's request for clearer advice to implementers, I would like 
to replace the above para with the following (Jukaa might be able to 
shorten it - he's better than me at brevity):

"If we now consider a hybrid forwarding system with transmission 
congestion dependent on the byte-size of packets but fixed-size 
buffers of one MTU per packet, the queue length can still simply be 
measured in bytes, based on the following reasoning.

When an interface buffers packets, we need to ask what is the root 
cause of the congestion: the line or the buffer? In this case, the 
line is the root of the congestion. Packets temporarily back-up into 
the buffer because more bits are arriving than departing. Even though 
a run of small packets would fill MTU-sized buffers more quickly, the 
line will also drain them more quickly. Therefore, the queue is 
always merely a symptom of how congested the line is. Certainly, once 
the buffer gets full, we could say the buffer has become congested 
too. But reducing the arriving bit-rate would clear congestion of the 
line, which in turn would empty the buffer.

Theoretically, congestion in this hybrid could be treated as the 
probability of congestion of the buffer conditional on congestion of 
the line, where the sum of bytes in the buffers measures congestion 
of the line while the length of the queue in packets measures 
congestion of the queue. In practice, however, just measuring the 
bytes queued gets at the root cause: congestion of the line.

If the queue were building up behind forwarding look-ups or a 
firewall, then the problem would be the number of packets headers, 
not bits. But if it is a queue into a transmission line, the problem is bits.

In summary, for these hybrid cases, whether to measure the queue 
length in bits or packets depends primarily on whether the process in 
front of the queue is bit-congestible (e.g. a transmission line) or 
packet-congestible (e.g. forwarding look-ups). The internals of the 
buffer can usefully be ignored.
"


> > The other is that it
>>only really applies to interfaces to a unique and non-blocking
>>channel, like a serial line. When the congestible interface or queue
>>is on a shared resource such as the fabric in an input-queued switch,
>>an 802.11 interface, etc, the mathematics really involves two queues
>>- the interface queues (in a statistical sense if not a hardware
>>version) for access to the channel, and the packet is enqueued toward
>>the interface, and both arrival/departure distributions are
>>important.
>>
>>Consider, for example, an AQM implementation on a WiFi channel. We
>>have all been on busy 802.11 networks; we have this experience in our
>>own history at the IETF. The dynamics are fairly strange. At the IP
>>layer, sessions are generally between hosts on the WiFi network and
>>hosts somewhere else. At the WiFi layer the AP is a proxy for the
>>remote hosts, and is therefore in essence an extremely busy member of
>>the WiFi system that otherwise operates in an approximation to a
>>round-robin fashion. The effect is to build a bufferbloat scenario
>>into the AP. Due to the shared nature of the medium and the fact that
>>retransmissions are driven by absolute time (an RTO happens after a
>>certain point in time), what one is really looking for is a packet's
>>queue occupancy being measured in time. Bytes are a reasonable analog
>>to time (which is what makes bytes reasonable for a bit-congestible
>>interface) for noncontested interfaces, but a shared interface can
>>fail to emit a transmission for an arbitrary time interval. So I
>>might find myself, if some variant on Blue is in use, bumping the
>>mark/drop probability when the queue becomes full and *also* any time
>>one dequeues a packet that has been waiting longer than some time
>>interval.

Yes, you have to find something that approximates to a model of the 
sequence of congested resources. In byte-pkt we decided not to go 
deeply into this as another example. We felt the previous hybrid 
example had already covered the byte-vs-pkt issue sufficiently, and a 
wireless example might say a lot more about how to design a wireless 
AQM, but not a lot more about byte-v-pkt. Instead we referred to a 
Mobicom paper by Vasilios Siris on this 
<http://tools.ietf.org/html/draft-ietf-tsvwg-byte-pkt-congest-05#ref-ECNFixedWireless> 
(see Section 4.1.2. Congestion Measurement without a Queue).

That reference focuses on CDMA, but I chose it because it combines 
many forms of congestion you can get on an e2e fixed/wireless path. 
You can find other papers, including one on correctly signalling WiFi 
congestion here:
<http://www.ics.forth.gr/netlab/future_wireless.html>


>This is the combined example I suggested above AFAICT.

Yup.


Bob


>Joe
>
>>As to the "SHOULD NOT" regarding configuration, this probably makes a
>>lot of sense in academic circles. Speaking as a vendor, I'm going to
>>do what my customers tell me to. I'm happy enough with a default
>>configuration following this recommendation, but if my customer wants
>>it different, I'm not going to slap his hands.
>>
>>As a side note, I would generally suggest that we step aside from
>>"RED" and talk about "AQM". Blue, SFQ-Blue, and AVP are AQM
>>algorithms that are likely superior to RED from an operational
>>viewpoint.
>
>________________________________________________________________
>Bob Briscoe,                                BT Innovate & Design