[tsvwg] Comments on draft-white-tsvwg-nqb-02

["Bless, Roland (TM)" <[e-mail]>]

Hi,

I'm quite late to the discussion and eventually took time to have quick read
over the draft, but I think it is rather vague at times and there is
quite some
work required to either get rid of implicit assumptions or to write them
down
explicitly.

First, some general comments:

  * I find the whole term "non-queue building flows" a bit confusing.
    What the draft actually means are application-limited, mostly low
    data rate
    flows (sparse flows in RFC8290). According to the draft, capacity
    seeking
    flows seem to _always_ be queue building.
    This is not true, because it depends on the particular congestion
    control
    and how the capacity seeking is done. For instance, TCP LoLa [1]
    flows are
    not queue building, but are capacity seeking nevertheless, i.e.,
    they will
    saturate the bottleneck link. Strictly speaking, they are building a
    _limited_
    standing queue.

  * The notion "queue building" is also somewhat fuzzy: do you mean
    flows that build
    increasing and unlimited standing queues (i.e., until buffer is
    completely filled) or
    also flows that are able to build limited standing queues? LoLa and
    BBR would fall
    into the latter class, where BBR's standing queue depends on "the BDP".

  * Moreover, even sparse flows can build a queue: assume we have a
    sparse flow
    arriving at a link that is already saturated by a single capacity
    seeking flow.
    So even if the sparse flow is only sending 0.1% of the link capacity,
    it will _cause_ a queue to build up (under the assumption that the
    other flow
    does not back off immediately). The only difference is that the
    large flow possesses
    at lot more data in the queue compared to the sparse flow. Similarly, if
    there are a lot of sparse flows saturating a bottleneck, they will
    also create a growing
    queue (until they reduce their sending rate).

  * It's not clear to me how the service rates between the NQB and QB queue
    are distributed by the scheduler. From the description in the draft,
    I have
    no clue what kind of scheduler would be a good fit for the desired
    behavior.

  * Would it be possible to let an NQB flow (or several) saturate the
    link? I think it would be a perfectly valid scenario to let
    high-data rate, low delay flows use the NQB queue.
    "NQB traffic is not rate limited or rate policed." suggests that.

  * Can a low-latency high data rate flow be in the NQB queue, e.g., a
    TCP LoLa
    flow? An NQB PHB would be nice for them, because they are neither
    suppressed
    nor delay-wise adversely affected by the (loss-based) queue filling
    flows.

  * I also second Dave Black's comments on looking at the PHB definition
    requirements. A PHB definition should define the behavioral
    characteristics
    and more details, see
    https://tools.ietf.org/html/rfc2474#section-5 and
    https://tools.ietf.org/html/rfc2475#section-3
    While the PHB definition should be more abstract, I find it useful
    to provide at least
    examples of the mechanisms that can be used for an implementation.

Some more text related comments:

 1.    "Active Queue Management (AQM) mechanisms (such as PIE [RFC8033],
       DOCSIS-PIE [RFC8034], or CoDel [RFC8289]) can improve the quality of
       experience for latency sensitive applications, but there are
       practical limits to the amount of improvement that can be achieved
       without impacting the throughput of capacity-seeking applications."
    I'm not sure what your are referring to here. My experience tells me
    that
    this is also highly related to the congestion control variant. Let's
    assume
    Cubic TCP for now: If only a few flows are present at the AQM
    bottleneck, then the
    utilization/throughput may be adversely affected, but if already a
    small number of flows is
    present, a good AQM achieves good utilization due to its
    desynchronization feature.

 2. "These applications do not make use of
       network buffers, but nonetheless can be subjected to packet delay and
       delay variation as a result of sharing a network buffer with those
       that do make use of them."
    These application are actually using the buffer too, but their
    proportion
    w.r.t. queued data is probably small compared to "queue building" flows.

 3. "Here the data rate is essentially limited by the Application itself."
    That would be good to mention earlier. Is this a requirement for NQB
    flows?
    As I wrote above: there are also elastic NQB flows possible, using
    all available
    capacity. Furthermore, a lot of app-limited NQB flows could also
    saturate
    a link, do you exclude this case?

 4. "but there are also application flows that may be in a gray area in
    between
    (e.g. they are NQB on higher-speed links, but QB on lower-speed links)."
    Is this the case I just described: enough sparse flows may saturate
    lower-speed
    links already? or do you think of something else?

 5. "As an answer the last question" => As an answer _to_ the last question

 6. "Thus, a useful property of nodes that support separate queues for
    NQB and QB
       flows would be..."
    It seems that supporting separate queues is a requirement for the
    PHB. If so,
    please state that.

 7. " and for QB flows, the QB queue provides better performance
       (considering latency, loss and throughput) than the NQB queue."
    I don't see how they achieve better latency, except in the case that
    they experience less packet loss and thus suffer from less
    retransmissions,
    but this may be quite situation dependent...

 8. Should starvation of QB flows be impossible?

 9. "and reclassify those flows/packets to the QB queue."
    This should probably be explicit about flow classification, because
    otherwise
    per-microflow reordering may also adversely affect the e2e
    performance. So
    probably there is a requirement to move all packets belonging to a flow
    consistently to the QB queue.

10. "This queue SHOULD support a latency-based queue protection mechanism"
    as others already pointed out: this seems to be an important
    requirement and
    therefore, you should either state how that is supposed to work or
    the corresponding
    requirements.

11. "NQB traffic is not rate limited or rate policed." what happens if
    the load in the
    NQB queue increases (e.g., not by queue building flows)? Will QB
    flows be starved?
    I think that some scheduler will determine the rate share between
    NQB/QB queues...

12. "The node supporting the NQB PHB makes no guarantees on latency or
    data rate for NQB marked flows"
    The EF PHB is also not giving actual guarantees other than "low
    latency, low jitter,
    low loss"...

13. "The choice of
       the 0x2A codepoint (0b101010) for NQB would conveniently allow a node
       to select these two codepoints using a single mask pattern of
       0b101x10."
    Diffserv codepoints are unstructured and should NOT be handled like
    this.
    RFC 2474 requires anyway that the "mapping of codepoints to PHBs
    MUST be configurable"

14. Section 6: a discussion is missing if intermediate domains/nodes do not
    support the proposed NQB PHB: the low latency property may get lost
    completely if it gets treated as default PHB etc. RFC 2474 states
    "Packets received with an unrecognized codepoint SHOULD be forwarded
       as if they were marked for the Default behavior (see Sec. 4), and
       their codepoints should not be changed."

15. "cable broadband services MUST be configured
       to provide a separate queue for NQB traffic that shares the service
       rate shaping configuration with the queue for QB traffic."
    I have no clue what that means...

16. Section 9 contains some "negative examples" or why existing solutions
    are not solving the problem. I think there are probably some
    hidden/implicit
    requirements that should be rather extracted from this section and put
    into the definition of this PHB. For example you seem to do not want
    "that each non-sparse flow gets an equal fraction of link bandwidth"...
    or that there is a difference between the "sparse flow" definition of
    FQ and your definition of NQB flows (which are probably sparse, but
    app-limited or latency-sensitive at least?). What about
    latency-sensitive
    high data rate flows?

17. "it places unnecessary restrictions on the scheduling between the
    two queues"
    so what is the requirement for your own approach then?

That's all for now, regards

 Roland

[1] Mario Hock, Felix Neumeister, Martina Zitterbart, Roland Bless: TCP
LoLa: Congestion Control for Low Latencies and High Throughput,
IEEE LCN 2017, http://ieeexplore.ieee.org/document/8109356/