[quicwg/base-drafts] Length-prefixes and flow control (#1432)

[Martin Thomson <[e-mail]>]

HTTP/QUIC has a particular example of a pattern that seems likely to be usable in many other contexts.  A block of data is preceded by a length field.  The receiver of that block might read the length field then await the arrival of the entire block before processing the block.  During that time, the block is left unread so that the data can sit in the receive buffer of the transport.  This ensures that the data counts towards the flow control limits of the receiver.  This can prevent the sender from adding more messages that would only consume additional memory at the receiver.

In an extreme case, this leads to a deadlock.  The receiver is waiting for the end of a block.  The sender cannot send the remainder of the block because it does not have enough flow control credit.

This impasse doesn't naturally resolve in this case.  Worse, even if flow control might allow a single stream to make progress because the receiver ensures that it provides credit for the largest possible message or more, multiple concurrent streams could each consume a small part of the available flow control credit without allowing any single one to make progress.

Ultimately, there isn't an easy fix for this.  We could recommend a new input for prioritization: favour the sending of data for streams that are in the middle of sending a chunk, but that requires the transport be informed of message boundaries.  Relying on RST_STREAM to cancel streams until the logjam clears might work in some cases, but we have some examples already of streams that cannot be safely cancelled.  So this will always be a danger.  We gain some insight into what is going on with BLOCKED and friends, but ultimately this is just *hard*.

Not sure how much advice we can give here, or whether saying anything is even needed.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1432