Re: [payload] Benjamin Kaduk's Discuss on draft-ietf-payload-flexible-fec-scheme-17: (with DISCUSS and COMMENT)

[Giridhar Mandyam <[e-mail]>]

Thanks Benjamin.  Apologies for the late reply, but we (the editors) were confirming the best response to your follow-up questions.

> Maybe my question was not phrased well.  IIUC, the repair packets can only cover real media source streams that are part of the same RTP session.
So in order for both audio and video to be available in the same RTP session as potential input to asingle repair packet stream, the audio and video would need to be together in the same RTP session; presumably that would involve getting demux'd based on payload type or SSRC, which is what I'm reading 3550 as saying to not do.  Am I just misunderstanding that the repair inputs must be part of the same RTP session?

In my interpretation, the issue revolves around what "demultiplexing" means in this context.  I do not believe it was meant to contemplate the recovery of packets from a repair stream, which involves more than just "demultiplexing" of audio and video payloads.  I don't believe Sec. 5.2 of RFC 3550 really applies.

> >       Payload Type: The (dynamic) payload type for the FEC repair packets is determined through out-of-band means.  [...]
> >      Is "(e.g., SDP)" applicable here?

Sorry I missed this in my original response.  Yes, it does make sense and I have added it.

>>> Any reason not to include "retransmit" and "fixed block" mnemonics for the 'R' and 'F' bits? 
>> Can you explain further?  The R/F definitions already include mention of retransmission and fixed FEC L/D.
> I was thinking that the first sentence could say '(R and F bits, for "retransmit" and "fixed block")' to (in effect) "name" the bits right away.

Proposed text:

4.2.2.  FEC Header of FEC Repair Packets

   The format of the FEC header has 3 variants, depending on the values
   in the first 2 bits (R and F bits) as shown in Figure 10.  Note that
   R and F stand for "retransmit" and "fixed block", respectively.

> >> Please include a note here that several fields (e.g., P, PT, etc.)  in the FEC header are not meant to be interpreted directly but are instead actual FEC parity data akin to the following "payload". (Absent such an indication, the reader could see that these fields are "used to determine" values when they appear to contain values directly, and get confused.)
> > >I would suggest adding a forward-reference to Section 6 since that describes how the Repair Payload is calculated.
>> I am sorry.  I do not understand what is meant by "used to determine" values.  Can you explain?
>I was trying to consolidate remarks about 4.2.2, 4.2.2.1 and 4.2.2.2 into a single comment, but it came across badly; sorry for the confusion.  The last bullet point of Section 4.2.2 notes that "The P, X, CC, M and PT recovery fields are used to determine the corresponding fields of the recovered packets". Similarly, Figures 12 and 13 both have fields named for "PT recovery", "length recovery", etc., with corresponding descriptions in the main text like "the length recovery (16 bits) field is used to determine the length of the recovered packets" -- these are the "used to determine" text to which I refer.  So my concrete suggestion is to add a reference to Section 6.3.2 at the end of that final bullet in Section 4.2.2.  I'd consider adding it to the field descriptions in 4.2.2.1 as well, but that might be overkill, so use your judgement.

Forward reference is added.  Thanks.

> >> Should implementations set bounds on L and D that are smaller than the maximum encodable value (255)?
> > Yes.  This is assumed.
>Perhaps it's best to state it explicitly rather than leaving an implicit assumption.

> >> If L=0, D=0, use the optional payload format parameters for L and D.  What is the behavior when those payload format parameters were not provided?
> >Optional payload format parameters may be provided in SDP (see Sec. 5.1).
>Right.  But suppose they weren't, and yet the L=0,D=0 packet shows up on the wire.  This is clearly an error condition; what's the error handling?

Proposed text immediately following Fig. 14:

   Given the 8-bit limit on L and D (as depicted in Figure 13), the
   maximum value of either parameter is 255.  The "optional payload
   format parameters" (Figure 14) to be used when L=0 and D=0 can be
   determined through out-of-band signaling (see Section 5).  If L=0 and
   D=0 and cannot be determined through out-of-band signaling, then the
   repair packet MUST be ignored by the receiver.  In addition when L=1
   and D=0, the repair packet becomes a retransmission of a
   corresponding source packet.

> >> The L=1 case seems to imply that some full packet retransmission will be used; is it worth calling that out as a consequence of such a parameter choice?
>> I am not sure that I understand this statement.  L=1 does not imply anything about the value of D.
>I agree with your statement.  But, we seem to allow for L to be 1, and in particular for L=1,D=0 (also L=1,D=1).  For those combinations of L and D, the "Row FEC" case will in fact be packet retransmission, as a degenerate case.  I'm proposing that either this oddity is mentioned explicitly (e.g., "note that when L=1, the Row FEC packets will just be retransmission"), or disallowed by the numerical constraints.  Though now that I know better how the multi-SSRC case is handled (per Magnus's comments and my reply thereto), I guess you could use L=1 as part of a multi-SSRC repair and it would not be equivalent to retransmission.  So the situation is more complicated than I first thought.
>Regardless, since the conditions here do allow L=1 as a possibility, I suggest some additional discussion in the text about how it's a bit strange and probably not expected to be used.

As you mentioned above for bundled protection, individual L/D combinations can be declared for each SSRC (such as L=1,D=0) but the overall repair packet may in fact not be a retransmission of any individual packet.  An example could be an audio/video session where very few audio packets (as few as one) are bundled with several video packets in the generation of the repair packet.  In such a case, it might be perfectly valid to set L=1, D=0 for the audio packet.   So I am not sure about what is best approach to the additional discussion.  Did you have a concrete suggestion?

>> > What is the interaction between rate, repair-window, and the L and D values?  That is, if we set L and D to be large, and rate to be small, can we end up claiming a repair window that is too small to accumulate the necessary L*D source packets and compute recovery packets?
>> Yes, it is possible.  We expect that specific uses of FLEX FEC will bound the appropriate values for repair window, L and D.  It is difficult to establish these bounds in this specification, however, since the applications that are currently making use of FLEX FEC are varied (e.g. WebRTC, 3GPP MTSI).  We expect these specification to provide concrete guidance on the expected repair windows, L and D, based on the target application.
>Please do not rely on unstated expectations for the behavior of consumers of this mechanism!  I strongly suggest to provide some indication, even a vague and incomplete one, that there are to be interrelations and implementation restrictions on these parameters.

Proposed additional section:


1.1.7.  Repair Window Considerations

   The value for the repair window duration depends on the L and D
   values and cannot be chosen arbitrarily.  More specifically, L and D
   values determine the lower limit for the repair window duration.  The
   upper limit of the repair window duration does not depend on the L
   and D values.

Note that a discussion of rate is difficult (in my opinion) without making assumptions about the source SSRC's.  For the above section, however, I can add a sentence such as "The rate of the source streams should also be considered, as the repair window duration should ideally span several packetization intervals in order to leverage the error correction capabilities of the parity code."  

>> > Section 5.2.1
>>>    o  The value for the repair-window parameter depends on the L and D values and cannot be chosen arbitrarily.  More specifically, L and D values determine the lower limit for the repair-window size. The upper limit of the repair-window size does not depend on the L and D values.
>>>Per my above remark, this consideration seems generally applicable and not limited to SDP Offer/Answer.
>> This is also covered in Sec. 1.1, which provides the general guidance.
>An inline note that the repair window "is defined as the time that spans a FEC block", I see now.  But this is still in fairly abstract terms; I would have expected to see the note I quoted in my ballot to appear in a top-level Section 5 entry, and not in the subsection specific to SDP O/A.

>>>    o  Any unknown option in the offer MUST be ignored and deleted from the answer.  If FEC is not desired by the receiver, it can be deleted from the answer.
>>> This sounds like it is restating an existing normative requirement  (in which case a reference and descriptive, non-normative, text seems appropriate).
>> This requirement is specific to SDP O/A. Can you explain further as to why you think there is a different normative requirement?
>Ali got what I intended, though I was hoping for a reference to the relevant SDP RFC as well as the s/MUST/must/.

Proposed modifications:

   o  The value for the repair-window parameter depends on the L and D
      values.  Based on the values of L and D, a lower bound on the
      repair-window can be inferred (see Section 1.1.7).

   o  Although combinations with the same L and D values but with
      different repair-window sizes produce the same FEC data, such
      combinations are still considered different offers.  The size of
      the repair-window is related to the maximum delay between the
      transmission of a source packet and the associated repair packet.
      This directly impacts the buffering requirement on the receiver
      side and the receiver must consider this when choosing an offer.

   o  Any unknown option in the offer must be ignored and deleted from
      the answer (see Section 6 of [RFC3264]).  If FEC is not desired by
      the receiver, it can be deleted from the answer.

Sec. 6.2:
> To be clear, this is an editorial matter and you are free to ignore me, but my suggested wording is "The first 16 bits of the RTP header (16 bits), though the first two (version) bits will be ignored by the recovery procedure".

Proposed text is added.

> s/compute the FEC bit string from/extract the FEC bit string as/

Proposed text is added.

>>> I don't see how this matches up with the bit string construction in  Section 6.2.
>> As per Sec. 6.2,  "The rest of the FEC bit string, which contains everything after ...
>In particular I'm confused about the order between length recovery and TS recovery.  In the FEC header (for non-retransmissions), length recovery appears before TS recovery.  In Section 6.2, as we extract things from the FEC bit string, we write out the length recovery value into the FEC header, and then (the next bits from the FEC bit string are) the TS recovery value.
>But here we take the TS field and then the length field from the recovery bit string, which is in the other order.  Am I missing some step that causes the order that these fields appear in the bit stream to change?

Agreed.  The proposed change that seems to fix this  in Sec. 6.3.2 is:

   11.  Set the SN field in the new packet to SEQNUM.

   12.  Take the next 16 bits of the recovered bit string and set the
        new variable Y to whatever unsigned integer this represents
        (assuming network order).  Convert Y to host order.  Y
        represents the length of the new packet in bytes minus 12 (for
        the fixed RTP header), i.e., the sum of the lengths of all the
        following if present: the CSRC list, header extension, RTP
        payload and RTP padding.

   13.  Set the TS field in the new packet to the next 32 bits in the
        recovered bit string.

   14.  Set the SSRC of the new packet to the SSRC of the missing source
        RTP stream.

>> "Given that FLEX FEC enables the protection of multiple source streams, there exists the possibility that multiple source buffers may be created that may not be used.  An attacker could leverage unused source buffers to as a means of occupying memory in a FLEX FEC endpoint.  ***In addition, an attack against the FEC parameters themselves (e.g. repair window, D or L values) can result in a receiver having to allocate source buffer space that may also lead to excessive consumption of resources.***  Moreover the application source data may not be perfectly matched with FLEX FEC source partitioning.  If this is the case, there is a possibility for unprotected source data if, for instance, the FLEX FEC implementation discards data that does not fit perfectly into its source processing requirements. " 

>That's an improvement, thanks.  I don't think it covers my penultimate paragraph about a network attacker being able to force bit flips in the recovered packet length field, though, and I do think it's worth documenting that as a risk (when integrity protection is not used).

  Given that FLEX FEC enables the protection of multiple source
   streams, there exists the possibility that multiple source buffers
   may be created that may not be used.  An attacker could leverage
   unused source buffers to as a means of occupying memory in a FLEX FEC
   endpoint.  In addition, an attack against the FEC parameters
   themselves (e.g. repair window, D or L values) can result in a
   receiver having to allocate source buffer space that may also lead to
   excessive consumption of resources.  ***Similarly, a network attacker
   could modify the recovery fields corresponding to packet lengths
   (assuming there are no message integrity mechanisms) which in turn
   could force unnecessarily large memory allocations at the receiver.***
   Moreover the application source data may not be perfectly matched
   with FLEX FEC source partitioning.  If this is the case, there is a
   possibility for unprotected source data if, for instance, the FLEX
   FEC implementation discards data that does not fit perfectly into its
   source processing requirements.

>I would strongly suggest adding some text at the end of or after the first pargaraph of section 4.2.2.2, to the effect of:

>  The values of L and D for a given block of recovery data will correspond to
  the type of recovery in use for that block of data.  In particular, for 2-D
  repair, the (L,D) values will not be constant across all packets for a
  given SSRC being repaired.  Similarly, the L and D values can differ across
  different blocks of repair data (repairing different SSRCs) in a single
  packet.

Proposed text has been added.  Thanks.

>Regardless, I am still not 100% clear on the layout of the repair payload for the multi-SSRC case.  E.g., in Figure 13, we see that there are multiple SN base/L/D entries before the payload, which is a single consolidated payload that (presumably) covers all the recovery SSRCs.
Your description above makes it sound like each SN base/L/D indicates a set of packets, and the payload is the XOR of all of those sets together (padded if necessary).  That is, one could perhaps imagine conceptually doing this in two steps: (1) XOR together the packets indicated by the respective SN base/L/D for each SSRC, and then (2) XOR those results together to combine the N SSRC recovery blocks into a single repair payload.  Did I get it right this time?

You can do it as you have described.  It is not strictly required to do it that way.  The source packets could be arranged in such a way that the XOR operation can be performed in one step, rather than the two you describe.  Any method is fine as long as the desired parity is created - row-wise, column-wise, or flex mask.



-----Original Message-----
From: Benjamin Kaduk <[e-mail]> 
Sent: Tuesday, February 26, 2019 4:26 PM
To: Giridhar Mandyam <[e-mail]>
Cc: Magnus Westerlund <[e-mail]>; The IESG <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [payload] Benjamin Kaduk's Discuss on draft-ietf-payload-flexible-fec-scheme-17: (with DISCUSS and COMMENT)

-------------------------------------------------------------------------
CAUTION: This email originated from outside of the organization.
-------------------------------------------------------------------------

On Mon, Feb 25, 2019 at 12:03:16AM +0000, Giridhar Mandyam wrote:
> Thanks to Magnus for the partial response, and Benjamin for the careful review.

Yes, a big thanks to Magnus -- the explanation was quite helpful!
I'll reply to his comments inline (i.e., at the very end of this message).

> >> It's a little odd to see so much content in Section 1.1 before we get to requirements notation and defintions/notations.
> 
> I don't know what to make of this comment.  Is there a concrete suggestion that you may have in mind?

To be clear, this matter is entirely editorial discretion and you should do what you prefer, without giving preference to my input.  But I would have taken all the 1.X subsections and put them into a new top-level section that appears between (the current) sections 3.2 and 4; "Conceptual Overview" would be a fine title for such a hypothetical new section.

> > I think I'm a bit confused about current best practices for  multiplexing, as RFC 3550 Section 5.2 says "separate audio and video streams SHOULD NOT be carried in a single RTP session and demultiplexed based on the payload type or SSRC fields", but we seem to be not only recommending using SSRC for demultiplexing repair packets, but also suggesting that the FEC can cover multiple different audio and/or video streams with different SSRCs. I guess RFC 8108 is supposed to clarify when it's okay to use multiple  SSRCs in the same RTP session, so maybe the answer is just "3550 was overly cautious and we don't worry about that anymore".
> 
> I think the confusion lies around audio and video source streams.  The repair packets corresponding to bundled protection are not really source streams, which is what the 3550 guidance was addressing.  I think 3550 did not contemplate bundled protection.

Maybe my question was not phrased well.  IIUC, the repair packets can only cover real media source streams that are part of the same RTP session.
So in order for both audio and video to be available in the same RTP session as potential input to asingle repair packet stream, the audio and video would need to be together in the same RTP session; presumably that would involve getting demux'd based on payload type or SSRC, which is what I'm reading 3550 as saying to not do.  Am I just misunderstanding that the repair inputs must be part of the same RTP session?

> > Section 4.2.1
> >
> >       Version (V) 2 bits: This MUST be set to 2 (binary 10), as this
> >       specification requires all source RTP packets and all FEC repair
> >       packets to use RTP version 2.  The reason for this restriction is
> >       the first 2 bits of the FEC header contain other information (R
> >       and F bits) rather than recovering the RTP version field.
> >
> > nit: is it better to say that the FEC mechanism does not recover 
> > this value, rather than talking about how the first 2 bits of the 
> > FEC header are used for something else?  (The FEC header's structure 
> > need not bear any relation to the 12-byte RTP header, AFAICT.)
> 
> Proposal:  remove the sentence starting with "The reason for this restriction ...".
> 
> >
> >       Payload Type: The (dynamic) payload type for the FEC repair
> >       packets is determined through out-of-band means.  [...]
> >
> > Is "(e.g., SDP)" applicable here?
> >
> >       Sequence Number (SN): The sequence number follows the standard
> >       definition provided in [RFC3550].  definition.  Therefore it 
> > must
> >
> > nit: drop separate "definition."
> >
> 
> Good catch.  Will drop in revised I-D.
> 
> >       multiplex multiple repair streams in an RTP session.  The repair
> >       streams' SSRC's CNAME SHOULD be identical to the CNAME of the
> >       source RTP stream(s) that this repair stream protects.  An FEC
> >       stream that protects multiple source RTP streams with different
> >       CNAME's uses the CNAME associated with the entity generating the
> >       FEC stream or the CNAME of the entity on whose behalf it performs
> >       the protection operation.  In cases when the repair stream covers
> >       packets from multiple source RTP streams with different CNAME
> >       values, any of these CNAME values MAY be used.
> >
> > I'm not sure I'm parsing this properly; the penultimate sentence 
> > says that the CNAME to use is determined by nature of the entity 
> > producing the repair stream, but the last sentence says that there is a nondeterministic choice.
> >
> 
> Proposed change to last sentence:  "in cases when the repair stream covers packets from multiple source RTP stream with difference CNAME values and none of these CNAM values can be associated with the entity generating the FEC stream, then any of these CNAME values MAY be used."

That works for me, as does the follow-up suggestion from Ali.

> > Section 4.2.2
> >
> > Any reason not to include "retransmit" and "fixed block" mnemonics 
> > for the 'R' and 'F' bits?
> >
> 
> Can you explain further?  The R/F definitions already include mention of retransmission and fixed FEC L/D.

I was thinking that the first sentence could say '(R and F bits, for "retransmit" and "fixed block")' to (in effect) "name" the bits right away.

> > Please include a note here that several fields (e.g., P, PT, etc.) 
> > in the FEC header are not meant to be interpreted directly but are 
> > instead actual FEC parity data akin to the following "payload".
> > (Absent such an indication, the reader could see that these fields are "used to determine"
> > values when they appear to contain values directly, and get 
> > confused.)
> >
> > I would suggest adding a forward-reference to Section 6 since that 
> > describes how the Repair Payload is calculated.
> 
> I am sorry.  I do not understand what is meant by "used to determine" values.  Can you explain?

I was trying to consolidate remarks about 4.2.2, 4.2.2.1 and 4.2.2.2 into a single comment, but it came across badly; sorry for the confusion.  The last bullet point of Section 4.2.2 notes that "The P, X, CC, M and PT recovery fields are used to determine the corresponding fields of the recovered packets". Similarly, Figures 12 and 13 both have fields named for "PT recovery", "length recovery", etc., with corresponding descriptions in the main text like "the length recovery (16 bits) field is used to determine the length of the recovered packets" -- these are the "used to determine" text to which I refer.  So my concrete suggestion is to add a reference to Section 6.3.2 at the end of that final bullet in Section 4.2.2.  I'd consider adding it to the field descriptions in 4.2.2.1 as well, but that might be overkill, so use your judgement.

> > Section 4.2.2.2
> >
> > Should implementations set bounds on L and D that are smaller than 
> > the maximum encodable value (255)?
> 
> Yes.  This is assumed.

Perhaps it's best to state it explicitly rather than leaving an implicit assumption.

> 
> > If L=0, D=0, use the optional payload format parameters for L and D.
> > What is the behavior when those payload format parameters were not 
> > provided?
> >
> 
> Optional payload format parameters may be provided in SDP (see Sec. 5.1).

Right.  But suppose they weren't, and yet the L=0,D=0 packet shows up on the wire.  This is clearly an error condition; what's the error handling?

> > The L=1 case seems to imply that some full packet retransmission 
> > will be used; is it worth calling that out as a consequence of such a parameter choice?
> 
> I am not sure that I understand this statement.  L=1 does not imply anything about the value of D.

I agree with your statement.  But, we seem to allow for L to be 1, and in particular for L=1,D=0 (also L=1,D=1).  For those combinations of L and D, the "Row FEC" case will in fact be packet retransmission, as a degenerate case.  I'm proposing that either this oddity is mentioned explicitly (e.g., "note that when L=1, the Row FEC packets will just be retransmission"), or disallowed by the numerical constraints.  Though now that I know better how the multi-SSRC case is handled (per Magnus's comments and my reply thereto), I guess you could use L=1 as part of a multi-SSRC repair and it would not be equivalent to retransmission.  So the situation is more complicated than I first thought.

Regardless, since the conditions here do allow L=1 as a possibility, I suggest some additional discussion in the text about how it's a bit strange and probably not expected to be used.

> 
> > Section 4.2.2.3
> >
> > nit: The "P|X" bits in Figure 15 seem indented by one too many spaces.
> 
> Thanks.  Will revise on next I.-D.
> 
> > Section 5.1 (all subsections)
> >
> > Having the ToP values for interleaved and non-interleaved 1-D 
> > protection presented in a different order than virtually all of the 
> > body text (that presents non-interleaved first) is needlessly hard on the reader.
> 
> What would you suggest?  Would sub-bullets help?

I don't think sub-bullets would make much of a difference.  I also don't want to suggest changing the actual protocol values at this late stage, so the choice would seem to be between a fairly drastic "swap sections 1.1.1 and 1.1.2, and similar changes throughougt the body text, 1.1.3, etc.", and leaving it as-is.  I expect I know which one you'll choose, though :)

> > What is the interaction between rate, repair-window, and the L and D 
> > values?  That is, if we set L and D to be large, and rate to be 
> > small, can we end up claiming a repair window that is too small to 
> > accumulate the necessary L*D source packets and compute recovery packets?
> 
> Yes, it is possible.  We expect that specific uses of FLEX FEC will bound the appropriate values for repair window, L and D.  It is difficult to establish these bounds in this specification, however, since the applications that are currently making use of FLEX FEC are varied (e.g. WebRTC, 3GPP MTSI).  We expect these specification to provide concrete guidance on the expected repair windows, L and D, based on the target application.

Please do not rely on unstated expectations for the behavior of consumers of this mechanism!  I strongly suggest to provide some indication, even a vague and incomplete one, that there are to be interrelations and implementation restrictions on these parameters.

> > Section 5.2.1
> >
> >    o  The value for the repair-window parameter depends on the L and D
> >       values and cannot be chosen arbitrarily.  More specifically, L and
> >       D values determine the lower limit for the repair-window size.
> >       The upper limit of the repair-window size does not depend on the L
> >       and D values.
> >
> > Per my above remark, this consideration seems generally applicable 
> > and not limited to SDP Offer/Answer.
> 
> This is also covered in Sec. 1.1, which provides the general guidance.

An inline note that the repair window "is defined as the time that spans a FEC block", I see now.  But this is still in fairly abstract terms; I would have expected to see the note I quoted in my ballot to appear in a top-level Section 5 entry, and not in the subsection specific to SDP O/A.

> >    o  Any unknown option in the offer MUST be ignored and deleted from
> >       the answer.  If FEC is not desired by the receiver, it can be
> >       deleted from the answer.
> >
> > This sounds like it is restating an existing normative requirement 
> > (in which case a reference and descriptive, non-normative, text 
> > seems appropriate).
> 
> This requirement is specific to SDP O/A. Can you explain further as to why you think there is a different normative requirement?

Ali got what I intended, though I was hoping for a reference to the relevant SDP RFC as well as the s/MUST/must/.

> > Section 6.2
> >
> >    o  The first 16 bits of the RTP header (16 bits).
> >
> > Maybe note here that we'll actually ignore the first 2 bits?
> 
> Why?  The FEC repair is covering all relevant parts of the first 16 bits of the RTP header.

You can only say that the FEC repair is covering all relevant parts because we require a specific RTP version and thus mak the first 2 bits not relevant!

To be clear, this is an editorial matter and you are free to ignore me, but my suggested wording is "The first 16 bits of the RTP header (16 bits), though the first two (version) bits will be ignored by the recovery procedure".

> > Section 6.3.2
> >
> >    2.   For the repair packet in T, compute the FEC bit string from the
> >         first 80 bits of the FEC header.
> >
> > I'm scratching my head a bit at this.  Is this operation something 
> > other than "take the first 80 bits of the FEC header"?  (If not, the 
> > length and sequence number base seem to be in different places in 
> > the source packets and FEC bit string, if I'm reading things right.)
> 
> Yes, this is simply the first 80 bits as per the header formats in Sec.'s 4.2.2.1 and 4.2.2.2.  The wording as it stands seems to be accurate.  Did you have a suggestion in mind?

s/compute the FEC bit string from/extract the FEC bit string as/

> >    11.  Set the SN field in the new packet to SEQNUM.  Skip the next 16
> >         bits in the recovered bit string.
> >
> > To be clear, we're skipping over the xor of the reconstructed length 
> > field with the seqnum field of the source packets?
> 
> Yes.
> 
> >    13.  Take the next 16 bits of the recovered bit string and set the
> >         new variable Y to whatever unsigned integer this represents
> >         (assuming network order).  Convert Y to host order.  Y
> >         represents the length of the new packet in bytes minus 12 (for
> >         the fixed RTP header), i.e., the sum of the lengths of all the
> >         following if present: the CSRC list, header extension, RTP
> >         payload and RTP padding.
> >
> > I don't see how this matches up with the bit string construction in 
> > Section 6.2.
> 
> As per Sec. 6.2,
> "The rest of the FEC bit string, which contains everything after
>       the fixed 12-byte RTP header of the source packet, is written into
>       the Repair "Payload" following the FEC header, where "Payload"
>       refers to everything after the fixed 12-byte RTP header, including
>       extensions, CSRC list, true payloads, and padding."

In particular I'm confused about the order between length recovery and TS recovery.  In the FEC header (for non-retransmissions), length recovery appears before TS recovery.  In Section 6.2, as we extract things from the FEC bit string, we write out the length recovery value into the FEC header, and then (the next bits from the FEC bit string are) the TS recovery value.
But here we take the TS field and then the length field from the recovery bit string, which is in the other order.  Am I missing some step that causes the order that these fields appear in the bit stream to change?

> > Section 6.3.3
> >
> >    1.  Append Y bytes to the new packet.
> > [...]
> >    5.  Append the recovered bit string (Y octets) to the new packet
> >        generated in Section 6.3.2.
> >
> > I think a different verb than "append" should be used in step 1, 
> > perhaps "allocate Y additional bytes for the new packet", as the 
> > text as-written has us appending 2*Y bytes, only Y of which have a value specified.
> 
> Agreed.  Proposed new wording for steps 1 and 5:
> 
> "1.  Allocate Y additional bytes for the new packet generated in Section 6.3.2."
> "5.  Set the last Y octets in the new packet to the recovered bit string." 

That works for me; thanks.

> > Section 9
> >
> >                                                                The main
> >    security considerations for the RTP packet carrying the RTP payload
> >    format defined within this memo are confidentiality, integrity and
> >    source authenticity.  Confidentiality is achieved by encrypting the
> >    RTP payload.  Integrity of the RTP packets is achieved through a
> >    suitable cryptographic integrity protection mechanism.  [...]
> >
> > This phrasing of "is achieved by" implies that the mechanisms for 
> > doing so are defined in this document, but that's not the case.  
> > Don't we really mean things like "Confidentiality can be provided by 
> > encrypting the RTP payload"?
> 
> Proposed new wording as recommended:  "Confidentiality can be provided 
> by encrypting the RTP payload."

Thanks!

> >    Given that FLEX FEC enables the protection of multiple source
> >    streams, there exists the possibility that multiple source buffers
> >    may be created that may not be used.  An attacker could leverage
> >    unused source buffers to as a means of occupying memory in a FLEX FEC
> >    endpoint.  Moreover the application source data may not be perfectly
> >    matched with FLEX FEC source partitioning.  If this is the case,
> >    there is a possibility for unprotected source data if, for instance,
> >    the FLEX FEC implementation discards data that does not fit perfectly
> >    into its source processing requirements.
> >
> > I don't think this text quite covers the risks when interacting with 
> > an adversarial endpoint -- an attacker could try to advertise FEC 
> > schemes with large D and L and/or large repair windows, that cause 
> > the receiver to consume a lot of resources buffering packets that 
> > may be used as repair inputs.  Endpoints need to be aware of the 
> > risk when deciding whether to accept FEC streams, e.g., via SDP Offer/Answer.
> >
> > Similarly, a network attacker could modify the recovery fields 
> > corresponding to packet lengths (when integrity protection is not in 
> > play), to force large allocations on the receiver.  It's fairly 
> > likely that this doesn't even require knowing which source packet(s) 
> > will be lost, since length is a 16-bit field and the expected input 
> > values are not likely to have the high bit(s) set.
> >
> > The need for integrity protection on the SDP Offer/Answer exchange 
> > is probably sufficiently well-documented elsewhere that we don't 
> > need to reiterate it here.
> 
> The Section 8 guidance covers scenarios where the use of a given FEC configuration can result in no benefits in performance (or even degradation in performance).  If a receiver detects FEC parameters that are inconsistent with the nature of the source data or the transmission conditions, then the receiver could reject the offer (as per Sec. 6 of RFC 3264, "An offered stream MAY be rejected in the answer, for any reason").  However, additional guidance could improve the current text.
> 
> "Given that FLEX FEC enables the protection of multiple source streams, there exists the possibility that multiple source buffers may be created that may not be used.  An attacker could leverage unused source buffers to as a means of occupying memory in a FLEX FEC endpoint.  ***In addition, an attack against the FEC parameters themselves (e.g. repair window, D or L values) can result in a receiver having to allocate source buffer space that may also lead to excessive consumption of resources.***  Moreover the application source data may not be perfectly matched with FLEX FEC source partitioning.  If this is the case, there is a possibility for unprotected source data if, for instance, the FLEX FEC implementation discards data that does not fit perfectly into its source processing requirements. " 

That's an improvement, thanks.  I don't think it covers my penultimate paragraph about a network attacker being able to force bit flips in the recovered packet length field, though, and I do think it's worth documenting that as a risk (when integrity protection is not used).

<continued below with responses to Magnus's message>

> 
> -----Original Message-----
> From: payload <[e-mail]> On Behalf Of Magnus 
> Westerlund
> Sent: Thursday, February 21, 2019 7:09 AM
> To: Benjamin Kaduk <[e-mail]>; The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail]; 
> [e-mail]; [e-mail]
> Subject: Re: [payload] Benjamin Kaduk's Discuss on 
> draft-ietf-payload-flexible-fec-scheme-17: (with DISCUSS and COMMENT)
> 
> ----------------------------------------------------------------------
> ---
> CAUTION: This email originated from outside of the organization.
> ----------------------------------------------------------------------
> ---
> 
> Hi Benjamin,
> 
> I am not one of the authors, but one that have helped beating on this document in the WG, so I think I can answer your questions. I think the authors should check what I say and check the last part of the comments.
> 
> On 2019-02-18 22:53, Benjamin Kaduk wrote:
> > Benjamin Kaduk has entered the following ballot position for
> > draft-ietf-payload-flexible-fec-scheme-17: Discuss
> >
> > When responding, please keep the subject line intact and reply to 
> > all email addresses included in the To and CC lines. (Feel free to 
> > cut this introductory paragraph, however.)
> >
> >
> > Please refer to
> > https://www.ietf.org/iesg/statement/discuss-criteria.html
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://datatracker.ietf.org/doc/draft-ietf-payload-flexible-fec-sch
> > em
> > e/
> >
> >
> >
> > --------------------------------------------------------------------
> > --
> > DISCUSS:
> > --------------------------------------------------------------------
> > --
> >
> > I'm confused about some parts of how I'd implement this.
> > It's quite possible this is just my error, but I'm including this 
> > point in the Discuss section in case it's not.  This basically 
> > relates to how multiple recovery packets from a given FEC block get 
> > encoded and identified on the wire, but also how to populate the 
> > source block when multiple SSRCs are included.
> >
> > In short: suppose that I have D=3 and L=2.  I should expect 5 repair 
> > packets for the six source packets in a block; the scheme for 
> > determining what order to generate them in and what their contents 
> > are is fairly clear to me.  But how do I identify them on the wire?  
> > I'm assuming that the D and L on the wire are fixed values, since 
> > there's the possibility to only send zero on the wire and negotiate 
> > their values out of band.  It's a little less clear whether the "SN base"
> > fields are expected to be the same for all 5 recovery packets based 
> > on a given block, but if they do change then I'm not sure how I tell 
> > whether a given recovery packet is for a row or a column.  Is this 
> > supposed to be using the sequence number from the outer RTP header 
> > for packet ordering, and the implicit order for row/column FEC packets?
> > (It seems that in case of very bad packet loss and dynamic
> > L+D, the receiver could then get out of sync as to what the sequence 
> > L+number
> > is that corresponds to the start of a new batch of recovery blocks.)
> 
> So, if one are going to do a 2-D FEC code and have indicated that in the signaling, each repair packet is still either a row or column FEC. So a Row packet for your D=3 and L=2 2-D FEC configurations are going to say:
> 
> Sn base= i, L=2 D=1
> 
> The rest of the Row packets for this block are going to have:
> 
> Sn base=i+2, L=2 D=1
> 
> Sn base= i+4, L=2 D=1
> 
> Then the column packets
> 
> Sn base=i, L=2 D=3
> 
> Sn base=i+1, L=2 D=3
> 
> >From a receiver perspective you are not actually not caring about 
> >what
> block structures the sender uses. You anyway only can store received FEC packet in a receiver buffer for the stipulated time. When a repair packet comes in one checks if that repairs any loss, otherwise stores it in the buffer.
> 

Okay.  So some key takeaways here are that the 'L' and 'D' values will be different on the wire for row and column packets even on the same recovery stream.  I would strongly suggest adding some text at the end of or after the first pargaraph of section 4.2.2.2, to the effect of:

  The values of L and D for a given block of recovery data will correspond to
  the type of recovery in use for that block of data.  In particular, for 2-D
  repair, the (L,D) values will not be constant across all packets for a
  given SSRC being repaired.  Similarly, the L and D values can differ across
  different blocks of repair data (repairing different SSRCs) in a single
  packet.

I'd also recommend tweaking the text on page 23 about "[no] column FEC will follow"; perhaps "If L>0, D=0, indicates that Row FEC is in use, with no column FEC in use for this SSRC", "If L>0, D=1, indicates that this packet is Row FEC but that column FEC packets are interspersed in ths stream", and "If L>0, D>0, indicates that this packet is column FEC of every L packets in a group of D packets starting at the SN base.  Whether or not Row FEC packets will be interspersed cannot be determined from just this packet".

For the L>0, D=1 case, I'd also suggest

OLD:
             Hence, FEC = SN, SN+1, SN+2, ... , SN+(L-1), SN+L will be

             produced for each row.

             Then FEC = SN, SN+L, SN+2L, ..., SN+(D-1)L will be produced

             for each column.

             After all row FEC's have been sent, then the column FEC's

             will be sent.
NEW:
             Hence, FEC = SN, SN+1, SN+2, ... , SN+(L-1), SN+L will be
             produced for each row.

             Then FEC = SN, SN+L, SN+2L, ..., SN+(D-1)L will be produced
             for each column.  Note that the initial SN will be different
             for all pairs of packets other than the first row/first column
             packets!
             After all row FEC's have been sent, then the column FEC's
             will be sent, though reordering could cause the FECs to appear
	     in a different order at the receiver.

But the above are all just suggestions and I do not insist on any specific change.

I do, however, remain concerned about the case where L=0,D=0 on the wire and the optional payload format parameters are used to indicate L and D.
In that case, it seems impossible to do 2-D FEC, since there is no way to indicate whether a given L=0,D=0 packet is row or column repair.  I think that pure 1-D column and pure 1-D row are individually unambiguous, though.
Perhaps Section 6.3.1 should disclaim the possibility of 2-D repair for this case?


> > I also don't see how, for the case when there are multiple SSRCs, I 
> > know how many source packets to include from each SSRC in order to 
> > make up the D x L source block -- since Section 6.2's discussion 
> > lumps all the "source packets" together into a single set that get 
> > mutually xor'd, that seems to imply that the encoding is not "do 
> > recovery for SSRC1, do recovery for SSRC2, ..., concatenate them all".
> 
> Well, for each SSRC one follows the SN base and L and D parameters. This results in a number of packets that the XOR is performed over.

I think I overlooked the text "If there are multiple SSRC's protected by the FEC packet, then there will be multiple blocks of data containing an SN base along with L and D fields" on my first read.  Or perhaps I also had forgotten that the SSRCs being recovered are indicated as CSRCs in the recovery packet's header.

Regardless, I am still not 100% clear on the layout of the repair payload for the multi-SSRC case.  E.g., in Figure 13, we see that there are multiple SN base/L/D entries before the payload, which is a single consolidated payload that (presumably) covers all the recovery SSRCs.
Your description above makes it sound like each SN base/L/D indicates a set of packets, and the payload is the XOR of all of those sets together (padded if necessary).  That is, one could perhaps imagine conceptually doing this in two steps: (1) XOR together the packets indicated by the respective SN base/L/D for each SSRC, and then (2) XOR those results together to combine the N SSRC recovery blocks into a single repair payload.  Did I get it right this time?

> Does this help?

Yes, thank you!  Now I am at a point where I can actually ask useful questions instead of just flailing around :)

-Benjamin