Re: [conex] byte vs packet counting

[Fred Baker <[e-mail]>]

on the matter of bytes and packets, as I tried (apparently unsuccessfully) to say in the other email, what I am worried about first is time, which both bytes and packets are a readily-measured analog to. When I probed to see what interface someone knew about that sent messages as messages rather than a string of bits, I was told that WiFi measurably operates with an upper bound packet rate and is the poster child for packet-based interfaces. Great, but there is no single device that can perceive or measure it that way - wonderful academic observation, but I have no idea how to use it in an implementation. The point on a WiFi interface is neither that I have too many bytes or packets queued up; the point is that one of those is true in the context of a congested shared medium, and we *also* have to get control of the medium.

I find myself interested in bytes-as-an-analog-for-time on a non-shared interface and time on a shared interface, in both cases because I can measure them in an implementation.

On Nov 21, 2011, at 3:56 AM, Bob Briscoe wrote:

> Matt,
> 
> [Adding the ConEx list back in]
> 
> I think we're both now agreeing that the /goal/ should be byte-balance for ConEx auditing. Correct?
> 
> I am also in full agreement that this goal isn't always easy to meet if retrofitting ConEx onto an existing transport like TCP. It works if the packet sizes are regular, but if they are lumpy, the sender is likely to understate or overstate. And I agree that these aren't necessarily just corner-cases (well, I guess there is a large set of cases with regular packet sizes, but the corner is also quite large).
> 
> You ask what I think we should do about this. My view, we should:
> a) state what the goal should be for all transports (byte-balance)
> b) give advice on how to implement a TCP sender to do as well as it can
> c) run experiments and see whether the outcome is good enough, or if further protocol mechanism is needed.
> 
> Concerning (b) advice on the best that TCP implementations can do:
> - the TCP receiver (if optionally supporting ConEx) should suppress ACK delay whenever it detects congestion and instead give immediate feedback 
> - TCP sender can use ack sequence numbers to improve its guess of the required size of an re-echo-loss or re-echo-ECN
> - If the receiver is not ConEx-aware, a sender that knows it is sending lumpy packet sizes can introduce additional credit to cover any mistakes.
> 
> If a ConEx sender does unintentionally understate bytes of congestion, the auditor will discard some packets, then the sender will redress the balance with some more re-echo-loss and it should all heal itself, albeit having suffered some losses in the process.
> 
> Do you think this is sufficient (at least to run experiments to test whether it is)?
> 
> 
> 
> Bob
> 
> At 20:55 18/11/2011, Matt Mathis wrote:
>> So here is a pathology: I have an application that sends 15001 bytes (10 segments + 1 byte), followed by 1 byte beacons every 10 second for 100 seconds and then the entire pattern repeats (total of 10*1500 + 10*1 segments every 100 seconds).  
>> 
>> Now suppose there are ECN marks near the end of the burst on every repeat.   How does this get signaled, and what is the correct sender response?   Due to delayed ACKs it may be ambiguous exactly which segment was marked (e.g. was it 1500 or 1 bytes?)   Even if the sender knows for sure that it was the 1500 byte segment that was marked, it has to wait until the next burst to send the proper feedback.
>> 
>> The problem is the ACKs carry the count of the number of marked segments, not marked bytes.  Also the re-feed itself has no size, except the length of the segments carrying it.
>> 
>> What do you think should happen in this case?
>> (BTW modern persistent web protocols, such as SPDY do stuff like this all the time, although perhaps not as extreme as my example.   BGP also has streaming data with very irregular message sizes.)
>> 
>> This would be completely clear if the ACKs and re-feedback were counts of bytes.
>> 
>> Thanks,
>> --MM--
>> The best way to predict the future is to create it.  - Alan Kay
>> 
>> 
>> 
>> On Thu, Nov 17, 2011 at 7:07 PM, Matt Mathis <[e-mail]> wrote:
>> 
>> I'm sorry I did not make my self clear. I understand and agree with your argument.
>> 
>> The problem is when the forward path is carrying lumpy data : highly irregular segment sizes, typical of streaming media, p-http, or bgp at edge of the onset of congestion. When there is an ecn mark the ack  only indicates that a segment was marked and not its size. If the sender has a choice of segments to re-feedback it has has no idea which to choose.  Worse it
>> may not have a choice.   Furthermore under pathological conditions it will persistently get it wrong.  
>> 
>> I can explain better when I have a real. Keyboard. 
>> On Nov 17, 2011 10:02 PM, "Bob Briscoe" <[e-mail]> wrote:
>> Matt,
>> 
>> This issue has come up in at least two threads recently: "Congestion" vs. "Congestion Volume" and "byte-counting in conex-destopt"
>> 
>> Consider the buffer into the high speed line you were talking about this morning - it holds packets in equal MTU-sized packet buffers, whatever the packet size.
>> 
>> When it buffers packets, we need to ask what exactly is congested: the line or the buffer? In this case, the line is congested. Packets are temporarily in the buffer because more bits arrived than departed. The queue is merely a symptom of how congested the line is. Certainly, once the buffer gets full, we could say the buffer has become congested too. But the root of the problem is how fast the line can carry away the bits.
>> 
>> If the queue was building up behind forwarding look-ups or a firewall, then the problem would be the number of packets headers, not bits. But if it's a queue into a transmission line, the problem is bits.
>> 
>> The point is, whether to count bits or packets depends on the process /in front/ of the queue (whether its a bit transmission line or processing packet headers). The internals of the buffer itself are irrelevant.
>> 
>> Then the question is how prevalent are per-packet processes as sources of congestion? Answer: There seems to be good reason why per-packet congestion will remain in the minority relative to per-byte...
>> 
>> During the process of writing draft-ietf-byte-pkt-congest, all the machine design folk who were consulted said that machines are generally designed so that any per-packet-processing can cope with a workload at line rate consisting mostly of small packets.
>> 
>> IOW, machine designs tend to use bit-congestion to protect the packet-processor from congestion.
>> 
>> ____________________
>> For those who prefer an example, assume:
>> MTU,       M = 1,500B = 12,000b
>> Line rate, X =    48Gbps
>> 
>> [The numbers are chosen to make the maths easy, not to reflect typical scenarios. I'm going to work in bits not bytes from now on.]
>> 
>> Imagine at some time that 260 of these fixed-size packet buffers are full, with 10 large and 250 small packets.
>> 
>> packet size       | S    | 12,000b  |   480b   |
>> no. pkts buffered | N    |     10   |   250    |
>> buffer space used | NM   |    120kb |     3Mb  |
>> pkt bits buffered | NS   |    120kb |   120kb  |
>> time to drain all | NS/X |  2,500ns | 2,500ns  |
>> 
>> Although each small packet takes up the same space in the buffer as a large packet, it's faster to get rid of it. 250 small packets take as long to drain as 10 large packets.
>> 
>> 
>> Bob
>> 
>> 
>> ________________________________________________________________
>> Bob Briscoe,                                BT Innovate & Design 
>> 
> ________________________________________________________________
> Bob Briscoe,                                BT Innovate & Design
> 
> _______________________________________________
> conex mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/conex