Re: [nwcrg] [irsg] IRSG review request draft-irtf-nwcrg-coding-and-congestion-09

[Nicolas Kuhn <[e-mail]>]

Dear Spencer, all,

Thank you so much for this review that contributes a lot, not only on the
readability but also on structural aspects.
I hope we addressed your comments in the updated version of this draft
https://datatracker.ietf.org/doc/draft-irtf-nwcrg-coding-and-congestion/

Thanks a lot for your help !

More inline.

Nico - on the behalf of my co-authors

On Sat, Feb 12, 2022 at 1:16 AM Spencer Dawkins at IETF <
[e-mail]> wrote:

> Dear Draft Authors,
>
> On Thu, Dec 16, 2021 at 5:49 PM Colin Perkins <[e-mail]> wrote:
>
>> IRSG members,
>>
>> The NWCRG has requested that draft-irtf-nwcrg-coding-and-congestion-09 be
>> considered for publication as an IRTF RFC. To progress this draft, we now
>> need at least one IRSG member to volunteer to provide a detailed review of
>> the draft, as follows:
>>
>
> And that turned out to be me. Please see below following Colin's
> characterization of the IRSG review of drafts.
>
> > The purpose of the IRSG review is to ensure consistent editorial and
>> technical quality for IRTF publications. IRSG review is not a deep
>> technical review. (This should take place within the RG.) At least one IRSG
>> member other than the chair of the RG bringing the work forth must review
>> the document and the RG's editorial process.
>> >
>> > IRSG reviewers should look for clear, cogent, and consistent writing.
>> An important aspect of the review is to gain a critical reading from
>> reviewers who are not subject matter experts and, in the process, assure
>> the document will be accessible to those beyond the authoring research
>> group. Also, reviewers should assess whether sufficient editorial and
>> technical review has been conducted and the requirements of this process
>> document, such as those described in IRTF-RFCs have been met. Finally,
>> reviewers should check that appropriate citations to related research
>> literature have been made.
>> >
>> > Reviews should be written to be public. Review comments should be sent
>> to the IRSG and RG mailing lists and entered into the tracker. All IRSG
>> review comments must be addressed. However, the RG need not accept every
>> comment. It is the responsibility of the shepherd to understand the
>> comments and ensure that the RG considers them including adequate dialog
>> between the reviewer and the author and/or RG. Reviews and their resolution
>> should be entered into the tracker by the document shepherd.
>> >
>> > The IRSG review often results in the document being revised. Once the
>> reviewer(s), authors, and shepherd have converged on review comments, the
>> shepherd starts the IRSG Poll on whether the document should be published.
>
>
> OK, don't freak out.
>
> I really like this draft, and I think it's really good, and really needed.
> So, please try not to notice that I have nine pages of notes following, and
> just keep telling yourselves that Spencer is mostly making suggestions for
> readability.
>
> [NK] Thanks a lot for these relevant suggestions !


> One reason the review is nine pages long is that I tried to include both
> the paragraph I'm talking about and proposed text for you to consider. I
> hope most of my comments are self-explanatory, but if you have any
> questions, please ask. I'd like to be supportive for this draft, and help
> you get it through IRSG balloting.
>
> [NK] Thanks a lot for your help. This discussion that overlaps RG is
important IMHO to better contribute in each of the field.

> Best,
>
> Spencer
>
> Some of my comments make assumptions about who will read this (important)
> document in the future. Please read my comments through that lens, as part
> of considering whether any particular comment is helpful.
>
> The document is very careful to talk about “reliable transfer protocols
> such as TCP”, which is fine, but then shouldn’t the title be something like
> “Coding and congestion control in transfer protocols”?
>
> [NK] To be honest, I skipped this comment when I first read your comments
and went through them all one by one but forgot this one. I can update the
document with an update on the title if you think this matters. IMHO we
could even go for "Coding and congestion control". I think the current
title is fine but I would be happy to update it if needed.


> In the introduction,
>
>    Coding is a reliability mechanism that is distinct and separate from
>
>    the loss detection of congestion controls.  [RFC5681] defines the
>
>    loss-based congestion control of TCP; since FEC coding repairs such
>
>    losses, blindly applying it may easily lead to an implementation that
>
>    also hides a congestion signal from the sender.  It is important to
>
>    ensure that such information hiding does not occur.
>
> It might be helpful to point out to your readers that standardized TCPs
> don’t have any signal except loss, so that blindly applying FEC coding will
> hide the only congestion signal from the sender. Since you’re not limiting
> yourself to TCP, perhaps this would be better as
>
>    Coding is a reliability mechanism that is distinct and separate from
>
>    the loss detection of congestion controls.  Since FEC coding repairs
>
>    losses, blindly applying FEC may easily lead to an implementation that
>
>    also hides a congestion signal from the sender.  It is important to
>
>    ensure that such information hiding does not occur, because loss may be
>
>    the only congestion signal available to the sender (e.g. TCP [RFC5681]).
>
> [NK] Thanks. The text has been updated following your suggestion.


> This text is super helpful,
>
>    We consider an end-to-end unicast data transfer with FEC coding in
>
>    the application (above the transport), within the transport or
>
>    directly below the transport.  A typical scenario for the
>
>    considerations in this document is a client browsing the web or
>
>    watching a live video.
>
> but might be even more super helpful if it had pointers to the document
> sections that apply to each architecture. I was thinking about something
> like
>
>    We consider three architecture for end-to-end unicast data transfer:
>
>    - with FEC coding in the application (above the transport) (Section 3),
>
>    - within the transport (Section 4), or
>
>    - directly below the transport (Section 5).
>
>    A typical scenario for the
>
>    considerations in this document is a client browsing the web or
>
>    watching a live video.
>
> [NK] Thanks. The text has been updated following your suggestion.


> Could you provide a reference for ECN on first use (and thanks for
> expanding it on first use, of course)?
>
[NK] Done !

>
> Figure 1 is helpful, but a reader might be forgiven if they assumed that
> “FEC happens below CC”, and then were surprised to discover that’s not
> always the case. Figures 2 and 3 clearly show these channels as independent
> - perhaps rotating Figure 1 to match would be more helpful.
>
> [NK] I have split the figure in two separate figures and hope this is
clearer.


> Isn’t the observation about TCP in this text
>
>    o  'network information' (input control plane for the transport
>
>       including CC): refers not only to the network information that is
>
>       explicitly signaled from the receiver, but all the information a
>
>       congestion control obtains from a network (e.g., TCP can estimate
>
>       the latency and the available capacity at the bottleneck).
>
> true for any transfer protocol?
>
> [NK] I have removed the TCP example to make it more generic.


> This text in Section 2.2 seems pretty fundamental.
>
>    o  The transport layer may provide an unreliable transport service
>
>       (e.g.  UDP or DCCP [RFC4340
> <https://datatracker.ietf.org/doc/html/rfc4340>]) or a partially reliable
> transport
>
>       service (e.g.  SCTP with the partial reliability extension
>
>       [RFC3758 <https://datatracker.ietf.org/doc/html/rfc3758>] or QUIC
> with the unreliable datagram extension
>
>       [I-D.ietf-quic-datagram
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#ref-I-D.ietf-quic-datagram>]).
> Depending on the amount of redundancy
>
>       and network conditions, there could be cases where it becomes
>
>       impossible to carry traffic.
>
>   o  The transport layer may implement a retransmission mechanism to
>
>       guarantee the reliability of a data transfer (e.g.  TCP).
>
>       Depending on how the FEC and CC functions are scheduled (FEC above
>
>       CC, FEC in CC, FEC below CC), the impact of reliable transport on
>
>       the FEC reliability mechanisms is different.
>
> In the second bullet, I (as the reader) know what to do next (“keep
> reading”), and if “(FEC above CC, FEC in CC, FEC below CC)” was annotated
> with forward references to the corresponding sections, I’d know where to
> keep reading if I was looking at one particular architecture. But after
> reading the first bullet, I don’t know what to do next. Are unreliable or
> partially reliable transport services in scope for this document? If so, is
> any further guidance contained in the document? If so, a pointer here would
> be helpful.
>
> [NK] Thanks. The text has been updated following your suggestion. Pointers
have been added.


> At this point in the document, I realized I’d been assuming that
> reliability was a transport service thing, because I’m a transport guy, but
> “reliability” is ALSO a FEC thing. Perhaps this is worth explicitly noting?
> You are talking about coding and transport in the same sentence in the
> second paragraph of the introduction - perhaps something like (editing my
> previous suggestion for this paragraph, but I’m just changing the first
> sentence)
>
>    Coding and the loss detection of congestion controls are two distinct
>
>    and separate reliability mechanisms that is distinct and separate from
>
>    the loss detection of congestion controls.  Since FEC coding repairs
>
>    losses, blindly applying FEC may easily lead to an implementation that
>
>    also hides a congestion signal from the sender.  It is important to
>
>    ensure that such information hiding does not occur, because loss may be
>
>    the only congestion signal available to the sender (e.g. TCP [RFC5681]).
>
> [NK] Thanks. The text has been updated following your suggestion.


> I think it’s also correct to say that both transport services and FEC
> services may provide either full or partial reliability, and this is also
> distinct and separate from what the other reliability mechanism is
> providing, but I don’t think the RG has talked about that, so it doesn’t
> need to be in this document unless the RG thinks it should be.
>
> ISTM that Section 2.5 might be better placed earlier in Section 2, since
> this is talking about the reason we care about hiding congestion signals
> when a bottleneck is shared (if you don’t care about fairness, or
> qualifying or limiting harm, you don’t actually need to read this document
> :-)). It might also be appropriate to change the section title from
> “Fairness, a policy concern” to “Fairness, Quantifying and Limiting Harm,
> and Policy Concerns”.
>
> [NK] Thanks. The text has been updated following your suggestion. The
section has been moved earlier in the section with an updated title.


> In Section 3,
>
>    The advantage of this approach is that the FEC overhead does not
>
>    contribute to congestion in the network.  When congestion control is
>
>    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>    implemented at the transport layer, the repair symbols are sent
>
>    following the congestion window or rate determined by the CC
>
>    mechanism.  This can result in improved quality of experience for
>
>    latency sensitive applications such as VoIP or any not-fully reliable
>
>    services.
>
> This might be clearer as
>
>    The advantage of this approach is that the FEC overhead does not
>
>    contribute to congestion in the network when congestion control is
>
>    implemented at the transport layer, because the repair symbols are sent
>
>    following the congestion window or rate determined by the CC
>
>    mechanism.  This can result in improved quality of experience for
>
>    latency sensitive applications such as VoIP or any not-fully reliable
>
>    services.
>
> [NK] Thanks. The text has been updated following your suggestion.


> You might also want to expand VoIP here (it’s the first usage).
>
> [NK] Done !

> The next paragraph is perhaps too specific to QUIC datagrams and TCP?
>
>    This approach requires that the transport protocol does not implement
>
>    a fully reliable in-order data transfer service (e.g., like TCP).
>
>    QUIC with unreliable datagram extension [I-D.ietf-quic-datagram] is
>
>    an example of a protocol for which this is relevant.  In cases where
>
>    QUIC traffic is blocked and a fall-back to TCP is proposed, there is
>
>    a risk for bad interactions between TCP's full reliability and coding
>
>    schemes.  For reliable transfers, coding usage does not guarantee
>
>    better performance; instead, it would mainly reduce goodput.
>
> Perhaps
>
>    This approach requires that the transport protocol does not implement
>
>    a fully reliable in-order data transfer service (e.g., like TCP).
>
>    QUIC with unreliable datagram extension [I-D.ietf-quic-datagram
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#ref-I-D.ietf-quic-datagram>]
> is
>
>    an example of a protocol for which this is relevant.  In cases where the
>
>    partially reliable transport is blocked and a fall-back to a reliable
>
>    transport is proposed, there is
>
>    a risk for bad interactions between reliability at the transport level
>
>    and coding
>
>    schemes.  For reliable transfers, coding usage does not guarantee
>
>    better performance; instead, it would mainly reduce goodput.
>
> [NK] Thanks. The text has been updated following your suggestion.


> In Section 3.2,
>
>    The congestion control mechanism receives network packets and may not
>
>    be able to differentiate repair symbols from actual source ones.
>
> I’m trying to think of a case where a congestion control mechanism
> implemented at the transport level would be able to differentiate between
> source and repair symbols, unless the transport protocol is doing more than
> providing RFC 8095 transport services.
>
> [NK] Thanks. The text has been updated with a discussion on the fact that
this may require services that are not in RFC8095


> In Section 3.4, I’m not sure what the first sentence is adding - and I
> think it can be deleted without damage.
>
> [NK] Thanks. The text has been updated following your suggestion.


> Sections 3.5, 4.5, and 5.5 are all “On partial ordering”, and Sections
> 3.6, 4.6, and 5.6 are all “On partial reliability”. Is it possible to use a
> title that describes more clearly where this is showing up (transport
> protocol, FEC, the service being provided to the application, or something
> else)? Sections 3.7, 4.7, and 5.7 all tell me that (“On transport
> multipath”).
>
> [NK] Thanks. The text has been updated following your suggestion. We have
added more specific titles to these sections.


> In Section 4, the second paragraph is providing pretty compelling
> arguments for doing FEC in the transport. I understand this document to be
> a discussion of the potential architectural space, but both the Abstract
> and the Introduction also say
>
>    Another objective is to encourage
>
>    the research community to also consider congestion control aspects
>
>    when proposing and comparing FEC coding solutions in communication
>
>    systems.
>
> Is it possible that there are disadvantages to doing FEC in the transport,
> that should also be mentioned in Section 4? I’m guessing, but only
> guessing, that it’s easier to combine a codebase that does transport and a
> codebase that does FEC if you put one on top of the other, rather than
> dealing with the complexity of coding and testing for transport and FEC
> together - is there anything like that?
>
> [NK] Indeed. This was missing and some drawbacks have been added.


> If there are only advantages, perhaps this document should be encouraging
> the research community to focus on the “FEC in transport” architecture
> specifically?
>
> In Section 4.1,
>
>    The addition of coding within the transport may impact the congestion
>
>    control mechanism and hide congestion losses.  Specific interaction
>
>    between congestion controls and coding schemes can be proposed (see
>
>    Section 4.2
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.2>
> , Section 4.3
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.3>
> and Section 4.4
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.4>).
> If no specific
>
>    interaction is introduced, the coding scheme may hide congestion
>
>    losses from the congestion controller and the description of
>
>    Section 5
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-5>
> may apply.
>
> the first sentence and the last sentence are talking about the same thing.
> Perhaps deleting the first sentence
>
>    Specific interaction
>
>    between congestion controls and coding schemes can be proposed (see
>
>    Section 4.2
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.2>
> , Section 4.3
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.3>
> and Section 4.4
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.4>).
> If no specific
>
>    interaction is introduced, the coding scheme may hide congestion
>
>    losses from the congestion controller and the description of
>
>    Section 5
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-5>
> may apply.
>
> could be clearer?
>
> [NK] Thanks. The text has been updated following your suggestion.


> In Section 4.2,
>
>    The receiver can differentiate between source packets and repair
>
>    symbols.  The receiver may indicate both the number of source packets
>
>    received and repair symbols that were actually useful in the recovery
>
>    process of packets.
>
> is there anything that the sending congestion control implementation can
> do with this information? (if not, the section title, “Congestion control
> and recovered symbols”, might not be helpful).
>
> I’m not sure what the relationship between Section 4.2 and 4.3 is, for the
> sending congestion controller. Are these sections, taken together, saying
>
>
>    -
>
>    the receiver indicating the number of source packets received
>    -
>
>    the receiver indicating the number of useful repair symbols
>    -
>
>    the sender tuning the coding ratio
>    -
>
>    the sender congestion controller adapts its transmission rate?
>
>
> [NK] Thanks. These sections have been merged.


> I think Section 4.4
>


>
>    The sender may exploit the information given by the receiver to
>
>    reduce the number of useless repair symbols and the resulting goodput
>
>    reduction.
>
> is talking about a reduction in a reduction(!). Is this saying
>
>    The sender may exploit the information given by the receiver to
>
>    reduce the number of useless repair symbols, and improve goodput.
>
> [NK] Thanks. The text has been updated following your suggestion.


> In Section 4.5,
>
>    The application may require in-order delivery of packets.  In this
>
>    case, both FEC and transport layer mechanisms should guarantee that
>
>    packets are delivered in order.  If partial ordering is requested by
>
>    the application, both the FEC and transport could relax the
>
>    constraints related to in-order delivery: reordering mechanisms at
>
>    the receiver may not be necessary.
>
> is it correct to say that the reordering mechanisms at the receiver are
> not necessary when the application requests partial ordering? If there are
> conditions where the reordering mechanisms are necessary even though the
> application requested partial ordering, it might be good to explain that.
> Section 4.6 does a good job of explaining why an application requiring
> partial reliability might still affect FEC - I’m asking about the same
> thing for an application requiring partial ordering.
>
> [NK] Thanks. The text has been updated with some discussions on partial
reordering at the FEC and CC.


> In Section 5,
>
>    Figure 8 presents an architecture where FEC is applied end-to-end
>
>    below the transport layer, but above the link layer.  Note that it is
>
>    common to apply FEC at the link layer, where it contributes to the
>
>    total capacity that a link exposes to upper layers.  This application
>
>    of FEC is out of scope of this document.  This includes the use of
>
>    FEC on top of a link layer in scenarios where the link is known by
>
>    configuration.  In the scenario considered here, the repair symbols
>
>    are sent on top of what is allowed by the congestion control.
>
> I might suggest saying a couple of things differently.
>
>    Figure 8 presents an architecture where FEC is applied end-to-end
>
>    below the transport layer, but above the link layer.  Note that it is
>
>    common to apply FEC at the link layer on one or more of the links
>
>    that make up the end-to-end path. The application of FEC at the
>
>    link layer contributes to the total capacity that a link exposes
>
>    to upper layers, but may not be visible to either the end-to-end
>
>    sender or receiver, if the end-to-end sender and receiver are separated
>
>    by more than one link, and therefore is out of scope for this document.
>
> -> This includes the use of
>
>    FEC on top of a link layer in scenarios where the link is known by
>
>    configuration.  In the scenario considered here, the repair symbols
>
>    are not visible to the end-to-end congestion controller and may be
>
>    sent on top of what is allowed by the congestion control.
>
> Just checking - does “This” refer to “FEC at the link layer”? I think it
> does, but wanted to make sure.
>
> [NK] Thanks. The text has been updated following your suggestion.


> In this text,
>
>    Examples of the solution could be to add a given percentage of the
>
>    congestion window or rate as supplementary symbols, or to send a
>
>    fixed amount of repair symbols at a fixed rate.  The redundancy flow
>
>    can be decorrelated from the congestion control that manages source
>
>    packets: a separate congestion control entity could be introduced to
>
>    manage the amount of recovered symbols to transmit on the FEC
>
>    channel.  The separate congestion control instances could be made to
>
>    work together while adhering to priorities, as in coupled congestion
>
>    control for RTP media [RFC8699
> <https://datatracker.ietf.org/doc/html/rfc8699>] in case all traffic can
> be assumed to
>
>    take the same path, or otherwise with a multipath congestion window
>
>    coupling mechanism as in Multipath TCP [RFC6356
> <https://datatracker.ietf.org/doc/html/rfc6356>].  Another
>
>    possibility would be to exploit a lower than best-effort congestion
>
>    control [RFC6297 <https://datatracker.ietf.org/doc/html/rfc6297>] for
> repair symbols.
>
> I wasn’t sure how “introducing a separate congestion control entity”
> results in something different from Section 4 (except that there are now
> two congestion controllers). It sounds like you are assuming that the
> separate congestion controller is at the same level as the congestion
> controller that is managing source packets if the two congestion
> controllers are working together - am I missing something?
>
> [NK] You are definitely right. The paragraph has been copied/pasted in
both sections 4 and 5.


> In Section 5.4,
>
>    Useless repair symbols only impact the load on the network without
>
>    actual gain for the coded flow.  Using feedback signaling, FEC
>
>    mechanisms can measure the ratio between actually used and useless
>
>    symbols, and adjust the coding rate.
>
> “actually used” reads oddly here, and I’d suggest using quotes to show
> that it’s a complete thought. But perhaps this could be better as
>
>    Useless repair symbols only impact the load on the network without
>
>    actual gain for the coded flow.  Using feedback signaling, FEC
>
>    mechanisms can measure the ratio between the number of symbols that
>
>    were actually used and the number of symbols that useless, and
>
>    adjust the coding rate.
>

[NK] Thanks. The text has been updated following your suggestion.

>
> In Section 5.6,
>
>    The transport or application layer above the FEC channel may require
>
>    partial reliability only.  In this case, FEC may provide an
>
>    unnecessary service if it is not aware of the reliability
>
>    requirements.  Partial reliability impacts the type of FEC and type
>
>    of codec that can be used, such as discussed in Section 2.4
> <https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-2.4>
> .
>
> If a link layer is aware of the (“transport or application”) reliability
> requirements, that’s a truly impressive layering violation, and I’m
> guessing that’s not the common case. Perhaps inverting this would help?
>
>    The transport or application layer above the FEC channel may require
>
>    partial reliability only. FEC may provide an
>
>    unnecessary service unless it is aware of the reliability
>
>    requirements.  Partial reliability impacts the type of FEC and type
>
>    of codec that can be used, such as discussed in Section 2.4.
>
>
[NK] Thanks. The text has been updated following your suggestion.

In Section 5.7,
>
>    The transport may exploit multiple paths without the FEC channel
>
>    being aware of it.  This depends on whether FEC is applied to all
>
>    subflows or each of the subflows individually.  When FEC is applied
>
>    to all the flows, there is a risk for the coding rate to be
>
>    inadequate for the characteristics of the individual paths.
>
> is it correct to say
>
>
>
>    The transport may exploit multiple paths without the FEC channel
>
>    being aware of it.  If FEC is aware that multiple paths are in use,
>
>    FEC can be applied to all subflows as an aggregate, or to each of the
>
>    subflows individually. If FEC is not aware that multiple paths are in
>
>    use, FEC can only be applied to each subflow individually.
>
>    When FEC is applied
>
>    to all the flows as an aggregate, the varying characteristics of the
>
>    individual paths may lead to a risk for the coding rate to be
>
>    inadequate for the characteristics of the individual paths.
>
> [NK] Thanks. The text has been updated following your suggestion.


> In this text in 6.2.1,
>
>    [RFC8095] describes the mechanisms provided by existing IETF
>
>    protocols such as TCP, SCTP or RTP.  [RFC8406
> <https://datatracker.ietf.org/doc/html/rfc8406>] describes the variety
>
>    of coding techniques.  The important level of combinations makes the
>
>                               ^^^^^^^^^^^^^^^
>
>    determination of an optimum parameters derivation very complex.  This
>
>    depends on application requirements and deployment context.
>
> is it correct to say
>
>    [RFC8095] describes the mechanisms provided by existing IETF
>
>    protocols such as TCP, SCTP or RTP.  [RFC8406
> <https://datatracker.ietf.org/doc/html/rfc8406>] describes the variety
>
>    of coding techniques.  The number of combinations makes the
>
>    determination of an optimum parameters derivation very complex.  This
>
>    depends on application requirements and deployment context.
>
> [NK] Thanks. The text has been updated following your suggestion.

>
> _______________________________________________
> nwcrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/nwcrg
>