Re: [nwcrg] [irsg] IRSG review request draft-irtf-nwcrg-coding-and-congestion-09

[Spencer Dawkins at IETF <[e-mail]>]

Dear Draft Authors,

On Thu, Dec 16, 2021 at 5:49 PM Colin Perkins <[e-mail]> wrote:

> IRSG members,
>
> The NWCRG has requested that draft-irtf-nwcrg-coding-and-congestion-09 be
> considered for publication as an IRTF RFC. To progress this draft, we now
> need at least one IRSG member to volunteer to provide a detailed review of
> the draft, as follows:
>

And that turned out to be me. Please see below following Colin's
characterization of the IRSG review of drafts.

> The purpose of the IRSG review is to ensure consistent editorial and
> technical quality for IRTF publications. IRSG review is not a deep
> technical review. (This should take place within the RG.) At least one IRSG
> member other than the chair of the RG bringing the work forth must review
> the document and the RG's editorial process.
> >
> > IRSG reviewers should look for clear, cogent, and consistent writing. An
> important aspect of the review is to gain a critical reading from reviewers
> who are not subject matter experts and, in the process, assure the document
> will be accessible to those beyond the authoring research group. Also,
> reviewers should assess whether sufficient editorial and technical review
> has been conducted and the requirements of this process document, such as
> those described in IRTF-RFCs have been met. Finally, reviewers should check
> that appropriate citations to related research literature have been made.
> >
> > Reviews should be written to be public. Review comments should be sent
> to the IRSG and RG mailing lists and entered into the tracker. All IRSG
> review comments must be addressed. However, the RG need not accept every
> comment. It is the responsibility of the shepherd to understand the
> comments and ensure that the RG considers them including adequate dialog
> between the reviewer and the author and/or RG. Reviews and their resolution
> should be entered into the tracker by the document shepherd.
> >
> > The IRSG review often results in the document being revised. Once the
> reviewer(s), authors, and shepherd have converged on review comments, the
> shepherd starts the IRSG Poll on whether the document should be published.


OK, don't freak out.

I really like this draft, and I think it's really good, and really needed.
So, please try not to notice that I have nine pages of notes following, and
just keep telling yourselves that Spencer is mostly making suggestions for
readability.

One reason the review is nine pages long is that I tried to include both
the paragraph I'm talking about and proposed text for you to consider. I
hope most of my comments are self-explanatory, but if you have any
questions, please ask. I'd like to be supportive for this draft, and help
you get it through IRSG balloting.

Best,

Spencer

Some of my comments make assumptions about who will read this (important)
document in the future. Please read my comments through that lens, as part
of considering whether any particular comment is helpful.

The document is very careful to talk about “reliable transfer protocols
such as TCP”, which is fine, but then shouldn’t the title be something like
“Coding and congestion control in transfer protocols”?

In the introduction,

   Coding is a reliability mechanism that is distinct and separate from

   the loss detection of congestion controls.  [RFC5681] defines the

   loss-based congestion control of TCP; since FEC coding repairs such

   losses, blindly applying it may easily lead to an implementation that

   also hides a congestion signal from the sender.  It is important to

   ensure that such information hiding does not occur.

It might be helpful to point out to your readers that standardized TCPs
don’t have any signal except loss, so that blindly applying FEC coding will
hide the only congestion signal from the sender. Since you’re not limiting
yourself to TCP, perhaps this would be better as

   Coding is a reliability mechanism that is distinct and separate from

   the loss detection of congestion controls.  Since FEC coding repairs

   losses, blindly applying FEC may easily lead to an implementation that

   also hides a congestion signal from the sender.  It is important to

   ensure that such information hiding does not occur, because loss may be

   the only congestion signal available to the sender (e.g. TCP [RFC5681]).

This text is super helpful,

   We consider an end-to-end unicast data transfer with FEC coding in

   the application (above the transport), within the transport or

   directly below the transport.  A typical scenario for the

   considerations in this document is a client browsing the web or

   watching a live video.

but might be even more super helpful if it had pointers to the document
sections that apply to each architecture. I was thinking about something
like

   We consider three architecture for end-to-end unicast data transfer:

   - with FEC coding in the application (above the transport) (Section 3),

   - within the transport (Section 4), or

   - directly below the transport (Section 5).

   A typical scenario for the

   considerations in this document is a client browsing the web or

   watching a live video.

Could you provide a reference for ECN on first use (and thanks for
expanding it on first use, of course)?

Figure 1 is helpful, but a reader might be forgiven if they assumed that
“FEC happens below CC”, and then were surprised to discover that’s not
always the case. Figures 2 and 3 clearly show these channels as independent
- perhaps rotating Figure 1 to match would be more helpful.

Isn’t the observation about TCP in this text

   o  'network information' (input control plane for the transport

      including CC): refers not only to the network information that is

      explicitly signaled from the receiver, but all the information a

      congestion control obtains from a network (e.g., TCP can estimate

      the latency and the available capacity at the bottleneck).

true for any transfer protocol?

This text in Section 2.2 seems pretty fundamental.

   o  The transport layer may provide an unreliable transport service

      (e.g.  UDP or DCCP [RFC4340
<https://datatracker.ietf.org/doc/html/rfc4340>]) or a partially reliable
transport

      service (e.g.  SCTP with the partial reliability extension

      [RFC3758 <https://datatracker.ietf.org/doc/html/rfc3758>] or QUIC
with the unreliable datagram extension

      [I-D.ietf-quic-datagram
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#ref-I-D.ietf-quic-datagram>]).
Depending on the amount of redundancy

      and network conditions, there could be cases where it becomes

      impossible to carry traffic.

  o  The transport layer may implement a retransmission mechanism to

      guarantee the reliability of a data transfer (e.g.  TCP).

      Depending on how the FEC and CC functions are scheduled (FEC above

      CC, FEC in CC, FEC below CC), the impact of reliable transport on

      the FEC reliability mechanisms is different.

In the second bullet, I (as the reader) know what to do next (“keep
reading”), and if “(FEC above CC, FEC in CC, FEC below CC)” was annotated
with forward references to the corresponding sections, I’d know where to
keep reading if I was looking at one particular architecture. But after
reading the first bullet, I don’t know what to do next. Are unreliable or
partially reliable transport services in scope for this document? If so, is
any further guidance contained in the document? If so, a pointer here would
be helpful.

At this point in the document, I realized I’d been assuming that
reliability was a transport service thing, because I’m a transport guy, but
“reliability” is ALSO a FEC thing. Perhaps this is worth explicitly noting?
You are talking about coding and transport in the same sentence in the
second paragraph of the introduction - perhaps something like (editing my
previous suggestion for this paragraph, but I’m just changing the first
sentence)

   Coding and the loss detection of congestion controls are two distinct

   and separate reliability mechanisms that is distinct and separate from

   the loss detection of congestion controls.  Since FEC coding repairs

   losses, blindly applying FEC may easily lead to an implementation that

   also hides a congestion signal from the sender.  It is important to

   ensure that such information hiding does not occur, because loss may be

   the only congestion signal available to the sender (e.g. TCP [RFC5681]).

I think it’s also correct to say that both transport services and FEC
services may provide either full or partial reliability, and this is also
distinct and separate from what the other reliability mechanism is
providing, but I don’t think the RG has talked about that, so it doesn’t
need to be in this document unless the RG thinks it should be.

ISTM that Section 2.5 might be better placed earlier in Section 2, since
this is talking about the reason we care about hiding congestion signals
when a bottleneck is shared (if you don’t care about fairness, or
qualifying or limiting harm, you don’t actually need to read this document
:-)). It might also be appropriate to change the section title from
“Fairness, a policy concern” to “Fairness, Quantifying and Limiting Harm,
and Policy Concerns”.

In Section 3,

   The advantage of this approach is that the FEC overhead does not

   contribute to congestion in the network.  When congestion control is

   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

   implemented at the transport layer, the repair symbols are sent

   following the congestion window or rate determined by the CC

   mechanism.  This can result in improved quality of experience for

   latency sensitive applications such as VoIP or any not-fully reliable

   services.

This might be clearer as

   The advantage of this approach is that the FEC overhead does not

   contribute to congestion in the network when congestion control is

   implemented at the transport layer, because the repair symbols are sent

   following the congestion window or rate determined by the CC

   mechanism.  This can result in improved quality of experience for

   latency sensitive applications such as VoIP or any not-fully reliable

   services.

You might also want to expand VoIP here (it’s the first usage).

The next paragraph is perhaps too specific to QUIC datagrams and TCP?

   This approach requires that the transport protocol does not implement

   a fully reliable in-order data transfer service (e.g., like TCP).

   QUIC with unreliable datagram extension [I-D.ietf-quic-datagram] is

   an example of a protocol for which this is relevant.  In cases where

   QUIC traffic is blocked and a fall-back to TCP is proposed, there is

   a risk for bad interactions between TCP's full reliability and coding

   schemes.  For reliable transfers, coding usage does not guarantee

   better performance; instead, it would mainly reduce goodput.

Perhaps

   This approach requires that the transport protocol does not implement

   a fully reliable in-order data transfer service (e.g., like TCP).

   QUIC with unreliable datagram extension [I-D.ietf-quic-datagram
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#ref-I-D.ietf-quic-datagram>]
is

   an example of a protocol for which this is relevant.  In cases where the

   partially reliable transport is blocked and a fall-back to a reliable

   transport is proposed, there is

   a risk for bad interactions between reliability at the transport level

   and coding

   schemes.  For reliable transfers, coding usage does not guarantee

   better performance; instead, it would mainly reduce goodput.

In Section 3.2,

   The congestion control mechanism receives network packets and may not

   be able to differentiate repair symbols from actual source ones.

I’m trying to think of a case where a congestion control mechanism
implemented at the transport level would be able to differentiate between
source and repair symbols, unless the transport protocol is doing more than
providing RFC 8095 transport services.

In Section 3.4, I’m not sure what the first sentence is adding - and I
think it can be deleted without damage.

Sections 3.5, 4.5, and 5.5 are all “On partial ordering”, and Sections 3.6,
4.6, and 5.6 are all “On partial reliability”. Is it possible to use a
title that describes more clearly where this is showing up (transport
protocol, FEC, the service being provided to the application, or something
else)? Sections 3.7, 4.7, and 5.7 all tell me that (“On transport
multipath”).

In Section 4, the second paragraph is providing pretty compelling arguments
for doing FEC in the transport. I understand this document to be a
discussion of the potential architectural space, but both the Abstract and
the Introduction also say

   Another objective is to encourage

   the research community to also consider congestion control aspects

   when proposing and comparing FEC coding solutions in communication

   systems.

Is it possible that there are disadvantages to doing FEC in the transport,
that should also be mentioned in Section 4? I’m guessing, but only
guessing, that it’s easier to combine a codebase that does transport and a
codebase that does FEC if you put one on top of the other, rather than
dealing with the complexity of coding and testing for transport and FEC
together - is there anything like that?

If there are only advantages, perhaps this document should be encouraging
the research community to focus on the “FEC in transport” architecture
specifically?

In Section 4.1,

   The addition of coding within the transport may impact the congestion

   control mechanism and hide congestion losses.  Specific interaction

   between congestion controls and coding schemes can be proposed (see

   Section 4.2
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.2>
, Section 4.3
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.3>
and Section 4.4
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.4>).
If no specific

   interaction is introduced, the coding scheme may hide congestion

   losses from the congestion controller and the description of

   Section 5
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-5>
may apply.

the first sentence and the last sentence are talking about the same thing.
Perhaps deleting the first sentence

   Specific interaction

   between congestion controls and coding schemes can be proposed (see

   Section 4.2
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.2>
, Section 4.3
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.3>
and Section 4.4
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-4.4>).
If no specific

   interaction is introduced, the coding scheme may hide congestion

   losses from the congestion controller and the description of

   Section 5
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-5>
may apply.

could be clearer?

In Section 4.2,

   The receiver can differentiate between source packets and repair

   symbols.  The receiver may indicate both the number of source packets

   received and repair symbols that were actually useful in the recovery

   process of packets.

is there anything that the sending congestion control implementation can do
with this information? (if not, the section title, “Congestion control and
recovered symbols”, might not be helpful).

I’m not sure what the relationship between Section 4.2 and 4.3 is, for the
sending congestion controller. Are these sections, taken together, saying


   -

   the receiver indicating the number of source packets received
   -

   the receiver indicating the number of useful repair symbols
   -

   the sender tuning the coding ratio
   -

   the sender congestion controller adapts its transmission rate?


I think Section 4.4

   The sender may exploit the information given by the receiver to

   reduce the number of useless repair symbols and the resulting goodput

   reduction.

is talking about a reduction in a reduction(!). Is this saying

   The sender may exploit the information given by the receiver to

   reduce the number of useless repair symbols, and improve goodput.

In Section 4.5,

   The application may require in-order delivery of packets.  In this

   case, both FEC and transport layer mechanisms should guarantee that

   packets are delivered in order.  If partial ordering is requested by

   the application, both the FEC and transport could relax the

   constraints related to in-order delivery: reordering mechanisms at

   the receiver may not be necessary.

is it correct to say that the reordering mechanisms at the receiver are not
necessary when the application requests partial ordering? If there are
conditions where the reordering mechanisms are necessary even though the
application requested partial ordering, it might be good to explain that.
Section 4.6 does a good job of explaining why an application requiring
partial reliability might still affect FEC - I’m asking about the same
thing for an application requiring partial ordering.

In Section 5,

   Figure 8 presents an architecture where FEC is applied end-to-end

   below the transport layer, but above the link layer.  Note that it is

   common to apply FEC at the link layer, where it contributes to the

   total capacity that a link exposes to upper layers.  This application

   of FEC is out of scope of this document.  This includes the use of

   FEC on top of a link layer in scenarios where the link is known by

   configuration.  In the scenario considered here, the repair symbols

   are sent on top of what is allowed by the congestion control.

I might suggest saying a couple of things differently.

   Figure 8 presents an architecture where FEC is applied end-to-end

   below the transport layer, but above the link layer.  Note that it is

   common to apply FEC at the link layer on one or more of the links

   that make up the end-to-end path. The application of FEC at the

   link layer contributes to the total capacity that a link exposes

   to upper layers, but may not be visible to either the end-to-end

   sender or receiver, if the end-to-end sender and receiver are separated

   by more than one link, and therefore is out of scope for this document.

-> This includes the use of

   FEC on top of a link layer in scenarios where the link is known by

   configuration.  In the scenario considered here, the repair symbols

   are not visible to the end-to-end congestion controller and may be

   sent on top of what is allowed by the congestion control.

Just checking - does “This” refer to “FEC at the link layer”? I think it
does, but wanted to make sure.

In this text,

   Examples of the solution could be to add a given percentage of the

   congestion window or rate as supplementary symbols, or to send a

   fixed amount of repair symbols at a fixed rate.  The redundancy flow

   can be decorrelated from the congestion control that manages source

   packets: a separate congestion control entity could be introduced to

   manage the amount of recovered symbols to transmit on the FEC

   channel.  The separate congestion control instances could be made to

   work together while adhering to priorities, as in coupled congestion

   control for RTP media [RFC8699
<https://datatracker.ietf.org/doc/html/rfc8699>] in case all traffic can be
assumed to

   take the same path, or otherwise with a multipath congestion window

   coupling mechanism as in Multipath TCP [RFC6356
<https://datatracker.ietf.org/doc/html/rfc6356>].  Another

   possibility would be to exploit a lower than best-effort congestion

   control [RFC6297 <https://datatracker.ietf.org/doc/html/rfc6297>] for
repair symbols.

I wasn’t sure how “introducing a separate congestion control entity”
results in something different from Section 4 (except that there are now
two congestion controllers). It sounds like you are assuming that the
separate congestion controller is at the same level as the congestion
controller that is managing source packets if the two congestion
controllers are working together - am I missing something?

In Section 5.4,

   Useless repair symbols only impact the load on the network without

   actual gain for the coded flow.  Using feedback signaling, FEC

   mechanisms can measure the ratio between actually used and useless

   symbols, and adjust the coding rate.

“actually used” reads oddly here, and I’d suggest using quotes to show that
it’s a complete thought. But perhaps this could be better as

   Useless repair symbols only impact the load on the network without

   actual gain for the coded flow.  Using feedback signaling, FEC

   mechanisms can measure the ratio between the number of symbols that

   were actually used and the number of symbols that useless, and

   adjust the coding rate.

In Section 5.6,

   The transport or application layer above the FEC channel may require

   partial reliability only.  In this case, FEC may provide an

   unnecessary service if it is not aware of the reliability

   requirements.  Partial reliability impacts the type of FEC and type

   of codec that can be used, such as discussed in Section 2.4
<https://datatracker.ietf.org/doc/html/draft-irtf-nwcrg-coding-and-congestion-10#section-2.4>
.

If a link layer is aware of the (“transport or application”) reliability
requirements, that’s a truly impressive layering violation, and I’m
guessing that’s not the common case. Perhaps inverting this would help?

   The transport or application layer above the FEC channel may require

   partial reliability only. FEC may provide an

   unnecessary service unless it is aware of the reliability

   requirements.  Partial reliability impacts the type of FEC and type

   of codec that can be used, such as discussed in Section 2.4.

In Section 5.7,

   The transport may exploit multiple paths without the FEC channel

   being aware of it.  This depends on whether FEC is applied to all

   subflows or each of the subflows individually.  When FEC is applied

   to all the flows, there is a risk for the coding rate to be

   inadequate for the characteristics of the individual paths.

is it correct to say



   The transport may exploit multiple paths without the FEC channel

   being aware of it.  If FEC is aware that multiple paths are in use,

   FEC can be applied to all subflows as an aggregate, or to each of the

   subflows individually. If FEC is not aware that multiple paths are in

   use, FEC can only be applied to each subflow individually.

   When FEC is applied

   to all the flows as an aggregate, the varying characteristics of the

   individual paths may lead to a risk for the coding rate to be

   inadequate for the characteristics of the individual paths.

In this text in 6.2.1,

   [RFC8095] describes the mechanisms provided by existing IETF

   protocols such as TCP, SCTP or RTP.  [RFC8406
<https://datatracker.ietf.org/doc/html/rfc8406>] describes the variety

   of coding techniques.  The important level of combinations makes the

                              ^^^^^^^^^^^^^^^

   determination of an optimum parameters derivation very complex.  This

   depends on application requirements and deployment context.

is it correct to say

   [RFC8095] describes the mechanisms provided by existing IETF

   protocols such as TCP, SCTP or RTP.  [RFC8406
<https://datatracker.ietf.org/doc/html/rfc8406>] describes the variety

   of coding techniques.  The number of combinations makes the

   determination of an optimum parameters derivation very complex.  This

   depends on application requirements and deployment context.