Re: [AVTCORE] Éric Vyncke's Discuss on draft-ietf-avtcore-rtp-evc-06: (with DISCUSS and COMMENT)

["Eric Vyncke (evyncke)" <[e-mail]>]

Stephan

Thank you for your reply and please accept my apologies for a belated reply of mine.

See in-line for EV>

TL;DR: out of the 3 discuss points:

  1.  Was discussed, and I will clear it
  2.  Will be addressed in a revised I-D, then I am clearing it
  3.  Deserves more discussion, especially when you have text draft for it

For all other points: thanks in advance for the revised I-D.

Regards

-éric

From: Stephan Wenger <[e-mail]>
Date: Tuesday, 5 December 2023 at 18:56
To: Eric Vyncke (evyncke) <[e-mail]>, The IESG <[e-mail]>
Cc: [e-mail] <[e-mail]>, [e-mail] <[e-mail]>, [e-mail] <[e-mail]>, [e-mail] <[e-mail]>
Subject: Re: Éric Vyncke's Discuss on draft-ietf-avtcore-rtp-evc-06: (with DISCUSS and COMMENT)
Hi Éric,
Thanks for this comprehensive review.  Please see inline in blue.
S.


From: Éric Vyncke via Datatracker <[e-mail]>
Date: Tuesday, December 5, 2023 at 06:50
To: The IESG <[e-mail]>

.... %<....%<..... text elided.

# DISCUSS (blocking)

Do not worry :-)

As noted in https://www.ietf.org/blog/handling-iesg-ballot-positions/, a
DISCUSS ballot is a request to have a discussion on the following topics:

## Section 4.3.1

What is the expected decoder behaviour when DONL is present while it should not?

What is meant by `conditional`? Does it mean 'optional' ?
This language was inherited from RFC 7798 and the DONL concept and interleaving in general is known since RFC3984.  I have yet to hear about implementer confusion regarding the above points.
DONL is “conditionally” present according to the value of sprop-max-don-diff; see the paragraph just below figure 3.  I think that “conditional” describes its presence very well.
EV> fair enough, BTW, it took me a while to understand that all the parameters (e.g., sprop-max-don-diff) are actually negotiated. It would be nice to add some text around this feature in the terminology/introduction section.
As for what happens if a sender puts the two bytes of DONL on the wire even if, during signaling, it announced it wouldn’t do so: the receiver has to act on what it knows.  It was advised, through sprop-max-don-diff equal to 0 (or sprop-max-don-diff not present in the SDP, in which case it is inferred to be zero) that there is no DONL on the wire.  Therefore it interprets the two bytes following the payload header as the first two bytes of the video codec syntax.  As those two bytes are unlikely to follow the EVC syntax, the EVC decoder will do whatever it is designed to do when receiving a non-compliant bitstream.
EV> a little hand waving (pushing the hot potato to the upper layer ;-) ) but fair enough.
I do not believe this situation warrants a text change.


## Section 4.3.2

Should this be a "MUST" in `the value of Reserve and E Must be equal to 0` ?
Thanks for this catch.  Yes, “MUST”. Will fix.
EV> Thanks


What is the expected decoder behaviour when `APs MUST NOT be nested; ` is
violated ?
Ah!  Recursive nesting, again.  I recall discussion about recursive nesting in the runup to at least RFC 3984 and RFC 7798, and there may have been other instances.
Aggregation units are widely implemented in H.264 and H.265-based systems and I have not heard that this area is a sore spot in the real world.  I have also not seen anyone sending recursively nested APs.
EV> it is not because nobody has ever nested AP that the receiver behavior should not be specified though.
Recursive nesting is prohibited to a) discourage sender implementers from wasting bits for unnecessary control information, and b) minimize the burden of receiver implementers.  Recursive nesting can trivially be detected;  you cannot have a NAL unit with type 56 in an aggregation unit.  If you find one, then it’s an error condition and the reaction is left to the implementation and intentionally left unspecified, on the principle that error resilience and error tolerance are product differentiators.  A practical system has many options, some listed here in order of ease of implementation: 1) ignore the nested AP; 2) ignore the nested AP and report a “packet loss” to the decoder if there were such a thing in the API, 3) implement nested APs and extract the NAL units from the nested APs.
I prefer not to change the text in this context.

EV> I am afraid that listing the options (1 to 3 see above) is probably mandatory here.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------


# COMMENTS (non-blocking)

## Section 1

Please expand HEVC and AVC before first use.
Wilco.

## Section 1.1

Is `walled-garden implementations` well-known ? To be honest, this is the first
time I see "walled-garden" applied to "implementations".
It's intuitive for us.  “Closed platform implementations” sounds soooo yesterday.  Anyone else has an opinion?


## Section 1.1.4

Figure 1, any reason why the bit numbers are in a box ? Usually (see figure 2),
the bit numbers are 'naked', i.e., without a frame. Same comment for figure 10
later in the text.


That’s a carry-over from really old work, RFC3984 specifically.  Will fix by making them “naked”.

Should this I-D also specifies "nuh_reserved_zero_5bits MUST be set to 0" ?
No.  In the RTP payload NAL unit types, these bits were intentionally left undefined.  If MPEG were ever adding scalable profiles to EVC, these bits would be the layer-id.  I at least have the hope that existing payload implementations would work with layered bitstreams, although absent signaling, things like SFUs would likely be hard to implement.  More broadly, the design philosophy behind this payload format (and others where I was involved in) is to specify only what needs specifying, and leave the rest open.


## Section 4.1

As I am not familiar with the RTP packet structure, is it obvious for the
implementers how the values of P, X, CC, and CSRC are selected ?


The short answer to that question is “yes”, and the RTP spec is a normative reference, hence we ought to be safe.  No change to the text required.
The long answer: RTP has certain mechanisms related to its general operation and independent from the RTP payload specification in use.  All the field you mention belong to this category.  In short:

·         P indicates the presence of “padding” octets at the end of the packet (after the payload).

·         X indicates the presence of an RTP packet header extension that sits between the RTP header and the payload.

·         CSRC or contributing source denote the SSRC of an RTP stream that was input to an RTP mixer when this RTP packet stream was created by the mixer.

·         CC or contributing source count is a four bit integer indicating the presence of up to 15 such CSRC identifiers of 4 octets length in the RTP header.

## Section 4.3.1

Please expand/explain `DONL`.

I had no time to request the EVC document to check whether PayloadHdr in EVC
cannot be 56/57... I am therefore trusting the authors, WG, and responsible AD
on the absence of collision. It may be worth to add a sentence (perhaps in
section 4.3) stating that PayloadHdr for NUL will never be equal to 56 or 57.


Thank you for your diligence.  NAL unit types 56..62 are left unspecified in the EVC spec.  The word “unspecified”, in ISO/MPEG lingo, means that they are reserved not for future use of ISO but for use by other SDOs.

## Section 4.3.3

The use of DONL is really unclear to me (see above), so how can the presence of
DONL be detected in `The DONL field, when present,` ?


See above, though signaling of sprop-max-don-diff not equal to 0.

As the I-D forbids "atomic" fragment, suggest to use a normative "MUST NOT" in
`the Start bit and End bit must not both be set to 1 in the same FU header`.


Will add.

When can a receiver detect that there is a missing fragment ? If this is on a
time-out, then should there be some RECOMMENDED value ?
Each fragment is carried in exactly one RTP packet.  Detecting missing RTP packets is a function of the RTP main spec and operates through sequence numbering, in some cases in combination to timeouts.  The RTP spec contains some limited guidance on what timeout values make sense.
Should an RTP payload format give further advice to the implementer on sensible timeouts for the RTP main spec implementation?  We in AVT* have never (that I recall) done so.  If we do, we will have to think hard about what would be sensible.  Using EVC as an example, it can be run with widely varying frame rates—which can be dynamic throughout the RTP session—and with widely varying Group of Picture sizes (presence of resynchronization points where a decoder can recover).  Temporal scalability also comes to play here.  In terms of time, I can imagine EVC scenarios where a multi-second timeout may make sense, and others where the timeout would need to be in the lower tens of millisecond range.  It’s an implementation scenario dependent value.  Also, latency is a key product differentiator in video conferencing, the main application for RTP.  People who use RTP generally know what they are doing and guard their knowledge.  I’m not sure a standards body has a mission to take away such a product differentiator.


While I like the idea of processing the first n-1 fragments anyway, why not
extending this flexibility to the first n-m fragments (i.e., the last m
fragments were lost) ? Of course, the E-bit based system cannot differentiate
between n-1 and n-m; then, is it worth mentioning it in the text?


EVC (as other video coding specs) have mechanisms that allow recovery from losses of units of video (called slices in most environments).  A cooperating encoder can use slices if there were hope that the decoder can recover from individual losses.
Fragments, on the other hand, are designed for the use without cooperating encoders.  During the RFC3984 times, we had long discussions whether we should support fragments at all, as they have so many disadvantages of true media-aware (here: MTU-size aware) encoding.  We added them at the end because there are scenarios (pre-encoded bitstreams, for example), where the RTP stack has no control over the NAL unit size but still wants to conform with smaller MTU sizes, so to limit the IP-level fragmentation and resulting increase in packet losses.  However, bitstreams that do not take MTU sizes into account are generally unrecoverable until the next picture boundary—today even more so than back in the 2003-timeframe RFC3984/H.264 times.  There’s no point in adding implementation logic or complexity ot allow for partial recovering.  That’s why we settled on the S/E bit solution in RFC3984, and stuck with it ever since.
I suggest no change in the text or the protocol here.

## Section 5

Possibly due to my lack of real-time knowledge, but is there a risk to increase
the end-to-end latency is the encoder must wait for several NUL before sending
them ?


I assume you refer to interleaving (sprop-max-don-diff > 0).  Yes, there’s an increase in latency.  However, that may be compensated by better error resilience due to interleaving of packets, especially when you have observed (through packet loss analysis on the RTP level) that your network experiences heavy bursty packet losses.  The interplay between those two was subject to academic work a long time ago, and frankly, the DONL mechanism was inherited in slightly modified form from RFC3984, when interleaving vs. FEC vs. retransmission was a big thing.  That has died down.  However, I have seen interleaving in the wild on occasion way after that timeframe.  I don’t think it’s time to deprecate interleaving and its protocol support just yet.  It’s another tool in the toolbox.

## Section 13.1

Is SAP still in use ? Anyway, I do not think that RFC 2974 should be normative
as it is cited as an example, i.e., it should rather be informative.


AFAIK, no one has used SAP in ages.  Will fix.

# NITS (non-blocking / cosmetic)

## Section 4.3

s/The Three different payload/The *t*hree different payload/ ?
Will fix.