Re: [AVTCORE] WG Last Call: "RTP-mixer formatting of multi-party Real-time text" - SFU

[Gunnar Hellström <[e-mail]>]

Bernard,

I have prepared modifications to satisfy your comments about Keep-alive 
and SFU.

For keep-alive, section 3.3 is proposed to be:
"3.3.  Keep-alive

    After that, the transmitter SHALL send keep-alive traffic to the
    receiver(s) at regular intervals when no other traffic has occurred
    during that interval, if that is decided for the actual connection.
    It is RECOMMENDED to use the keep-alive solution from [RFC6263].  The
    consent check of [RFC7675] is a possible alternative if it is used
    anyway for other reasons."

For SFU, I propose to have the section about considered solutions in 
section 1.2 on SFU to be:

"

    Multiple RTP streams, one per participant.
       One RTP stream per source would be sent in the same RTP session
       with the "text/red" format.  From some points of view, use of
       multiple RTP streams, one for each source, sent in the same RTP
       session would be efficient, and would use exactly the same packet
       format as [RFC4103] and the same payload type.  A couple of
       relevant scenarios using multiple RTP-streams are specified in
       "RTP Topologies" [RFC7667].  One possibility of special interest
       is the Selective Forwarding Middlebox (SFM) topology specified in
       RFC 7667 section 3.7 that could enable end to end encryption.  In
       contrast to audio and video, real-time text is only transmitted
       when the users actually transmit information.  Thus an SFM
       solution would not need to exclude any party from transmission
       under normal conditions.  In order to allow the mixer to convey
       the packets with the payload preserved and encrypted, an SFM
       solution would need to act on some specific characteristics of the
       "text/red" format.  The redundancy headers are part of the
       payload, so the receiver would need to just assume that the
       payload type number in the redundancy header is for "text/t140".
       The characters per second parameter (CPS) would need to act per
       stream.  The relation between the SSRC and the source would need
       to be conveyed in some specified way, e.g. in the CSRC. Recovery
       and loss detection would preferably be based on sequence number
       gap detection.  Thus sequence number gaps in the incoming stream
       to the mixer would need to be reflected in the stream to the
       participant and no new gaps created by the mixer.  However, the
       RTP implementation in both mixers and endpoints need to support
       multiple streams in the same RTP session in order to use this
       mechanism.  For best deployment opportunity, it should be possible
       to upgrade existing endpoint solutions to be multi-party aware
       with a reasonable effort.  There is currently a lack of support
       for multi-stream RTP in certain implementation technologies.  This
       fact made this solution only briefly mentioned in this document as
       an option for further study.

"

(I wonder if I should delete "in the same RTP session", because it seems 
that the SFM description in RFC 7667 does not specify that the streams 
to the participants must be in the same session.)   ????

In order for the specified negotiation to not preclude addition of other 
multi-party methods, e.g. an SFU based, I suggest to move the 
offer/answer section from section 3 to 2, and word it so that other 
multi-party methods are possible.

I also propose to change the name of the main method to be the 
"RTP-mixer based multi-party method"

With that, I hope that I have answered all WGLC comments to 
satisfaction, and I have a corresponding version -12 of the draft prepared.

Regards

Gunnar

-- 
Gunnar Hellström
GHAccess
[e-mail]

Den 2020-12-10 kl. 17:49, skrev Gunnar Hellström:
>
> After looking a bit more on the multi-stream issue, I want to reduce 
> the ambition I announced below.
>
> We should not put too many varied methods in one draft. It will be too 
> complicated to require or state compliance to them then.
>
> So, I want to back off to mention the possible benefits of 
> multi-stream RTP and SFU. I want to indicate that it would be possible 
> to negotiate between the specified RTP-mixer method and other 
> multi-party aware methods, but say that it is for further study to 
> specify any such method in more detail.
>
> This conclusion also matches the need from referencing from NENA 
> specifications.
>
>
> /Gunnar
>
> Den 2020-12-09 kl. 20:48, skrev Gunnar Hellström:
>>
>> Bernard,
>>
>> I am editing the draft to include a specification of use of multiple 
>> RTP streams in one RTP session to each participant. There seems to be 
>> so many ways to negotiate and use such topology, so I do not think I 
>> should dictate to use one specific. Therefore I suggest that the 
>> sections about that topology is made general and need exact 
>> specification at implementation time.
>>
>> The idea with the other methods (the RTP-mixer method and the method 
>> for multi-party unaware endpoints) are intended to be sufficiently 
>> specified to result in interoperability, e.g. between emergency 
>> services and mobile services.
>>
>> I do not see the same possibility for the multi-stream method. It is 
>> likely that RTT will be included in an environment where the other 
>> media and the application calls for use of specific RTP topologies 
>> and specific ways to negotiate the use of it. Therefore I intend to 
>> write a short description with  general statements and references to 
>> which considerations from the RTP-mixer method should be valid also 
>> for the RTP multi-stream methods.
>>
>> I intend to also check and adjust the use of terms as "multi-party 
>> aware" to make clear when it means one or both multi-party aware 
>> methods.
>>
>> A couple of further notes inline. I got some help to understand more;
>>
>> Den 2020-12-08 kl. 23:45, skrev Gunnar Hellström:
>>>
>>> Bernard, some proposed conclusions below:
>>>
>>> Den 2020-12-08 kl. 20:10, skrev Bernard Aboba:
>>>> Gunnar asked:
>>>>
>>>> "Questions:
>>>>
>>>> 1. Is 3GPP TS 26.114 Annex T an example of sdp for the kind of SFU
>>>> described in RFC 7667 section 3.7, with one media section per stream in
>>>> a lower number of streams than participants? If not, how does a normal
>>>> sdp look for an SFU?
>>>>
>>>> [BA] Annex T appears to use payload multiplexing (one payload type 
>>>> per stream) for simulcast, which is not commonly deployed.
>>>> It is more common to use SSRC multiplexing (one SSRC per stream), 
>>>> all on the same payload type.  Here are examples of what simulcast 
>>>> SDP looks like in various browsers:
>>>> A playground for Simulcast without an SFU - webrtcHacks 
>>>> <https://webrtchacks.com/a-playground-for-simulcast-without-an-sfu/>
>>>>
>>> [GH] Thanks. That example was with Bundle and WbRTC. If that is the 
>>> environment you imagine, we will not be allowed to use RTP transport 
>>> for RTT. Instead it is the data channel usage for RTT in RFC 8865. 
>>> Only audio and video are using RTP.
>>>
>>> But I assume that there could be use of the SFU technology also for 
>>> traditional SIP. What we need to consider for the current draft is 
>>> then if an sdp indicating capability for use of an SFU or some other 
>>> form of multi-stream RTP will have something characteristic for that 
>>> capability. E.g. an m-section for each stream.
>>>
>>> I think it is quite likely that it must be visible in the sdp that a 
>>> party has multi-stream RTP capability.
>>>
>>> Then the "rtt-mix" attribute is sufficient to indicate capability 
>>> for the RTP-mixer based multi-party solution, and we only need to 
>>> tell that other solutions may be implemented.
>>>
>>> I need to check wording so that the draft does not preclude other 
>>> multi-party RTP based RTT solutions.
>>>
>> [GH]I will just assume that it is visible in the sdp that a 
>> multi-party session with RTP multi-stream topology is offered, or a 
>> capability to use it. And that a party who is not capable of that 
>> method can indicate that fact in an offer or answer. I will not say how.
>>>
>>>>
>>>>
>>>> 2. RFC 7667 section 3.7 seems to tell that the sequence number is
>>>> regenerated in sequence by the middlebox. Is that right? That would 
>>>> make
>>>> it not possible to detect if total loss of text occurred. Recovery can
>>>> be done based on timestamp analysis, but not detection of unrecoverable
>>>> loss.
>>>>
>>>> [BA] The sequence number needs to be regenerated when an SFU 
>>>> switches between simulcast streams sent by a participant, sending 
>>>> only a single stream onward to the viewer.  This is needed since 
>>>> each simulcast stream has its own sequence number space, and 
>>>> endpoints typically do not support receiving simulcast.
>>>
>>> There are regulatory requirements to detect and mark cases of 
>>> suspected text loss. So, if there is packet loss from a participant 
>>> to the middlebox, that sequence gap must not be just ignored and 
>>> resulting in an unbroken sequence number series after the SFU. Can 
>>> the sequence number from the source be copied to the transmission 
>>> from the SFU?
>>>
>>> Does not video also have that need?
>>>
>> [GH]During active transmission from one source, the sequence number 
>> needs to step up at the same rate as the sequence numbers of incoming 
>> packets, so that gaps from the leg between the transmitter to the 
>> middlebox can be revealed and acted on by the receiving party. So, 
>> the offset adjustment is only intended to make the sequence look 
>> consequtive even if there was a pause in transmission of a source. 
>> Then it works for total loss detection for RTT.
>>>
>>>>
>>>> However an RTT sender would not send simulcast (e.g. multiple 
>>>> versions of the same text stream).  So the SFU would potentially 
>>>> forward only a single stream per participant, with no CSRCs. The 
>>>> question is whether a "multi-party aware endpoint" would be 
>>>> prepared to receive multiple SSRCs (and no CSRCs), with each SSRC 
>>>> representing a single RTT source.
>>> In order to have interoperability, the "rtt-mix" attribute must only 
>>> mean the RTP-mixer based solution. Both mixers and endpoints could 
>>> also have multi-stream capability, but the intention to open a 
>>> multiparty session with multi-stream need to have its own 
>>> characteristics.
>> [GH]I will differentiate the terms. Most likely so that multi-party 
>> aware means any of the two methods RTP-mixer and multi-stream RTP. 
>> And then specific wording for each method when there is talk about 
>> only one method.
>>>>
>>>> 3. How is the source conveyed? There is apparently an SSRC to SSRC
>>>> mapping taking place. Is there a lower number of SSRCs used in
>>>> transmission from the middlebox than the total number of participants?
>>>> If so, how is switch of source indicated? By RTCP?
>>>>
>>>> [BA] I am suggesting a scenario where there would be one SSRC per 
>>>> source.  The SFU could perhaps replace the SSRC received from a 
>>>> participant with an SSRC of its own, but each text stream would 
>>>> have a unique SSRC up to the maximum participant limit.
>> [GH]But the mapping between NAME and SSRC is lost when RTCP SDES is 
>> just sent along with replaced SSRC.  But I assume that there are 
>> methods to sort that out by some signaling that I dont know. So since 
>> I will not be specific I will not bother about such detail.
>>>>
>>>>
>>> RTT is specific in that no data flows when the user does not send 
>>> any information. In the other media there is always contents. 
>> [GH] So in a way it can be said to be "SFU by nature".   I might 
>> mention that. It eases switching. In the normal case no switching 
>> will be needed. All simultaneous senders can likely always be allowed 
>> to transmit on to the receivers.
>>>
>>>>
>>>> If we find that it is likely possible to use the SFU, then I would
>>>> anyway think that we just include general hints of its use in this
>>>> draft, and let further work detail it. Would that be an acceptable
>>>> conclusion?
>> [GH]I am continuing on that line.
>>>>
>>>> [BA] My question was whether a "multi-party aware" endpoint could 
>>>> support an SFU (SSRCs but no CSRCs), and how the negotiation would 
>>>> work (whether it was supported or not).  If it can work and you can 
>>>> indicate how it would be done (or how it would fail if it cannot be 
>>>> done), that would be fine with me.
>>>> "
>>>
>>> I need another example of sdp for an SFU session establishment in a 
>>> traditional SIP environment to be able to answer.
>>>
>> [GH] I changed my mind. My general approach does not need any more 
>> sdp examples.
>>>
>> Thanks,
>>
>> Gunnar
>>
>>>
>>> Thanks,
>>>
>>> Gunnar
>>>
>>>>
>>>> On Mon, Dec 7, 2020 at 3:03 PM Gunnar Hellström 
>>>> <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>>     Bernard, I indicated that I would like to discuss the SFU.
>>>>
>>>>     You said: "Also, some clarifications of desired behavior by
>>>>     SFUs might
>>>>     be helpful." and mentioned SFU in a qouple of other comments.
>>>>
>>>>     I like the idea to have a possibility to use E2E encryption. I
>>>>     also
>>>>     imagined that with separate RTP streams it would be possible to
>>>>     better
>>>>     detect if unrecoverable loss of text appears or not after some
>>>>     lost
>>>>     packets. But I get unsure if that is possible when I read
>>>>     section 3.7 of
>>>>     RFC 7667.
>>>>
>>>>     I need to understand the SFU description in RFC 7667 better to
>>>>     judge if
>>>>     it really can carry RFC 2198 coded text with maintained
>>>>     possibility to
>>>>     detect unrecoverable loss.
>>>>
>>>>     Before IETF 108, the draft had both an RTP multi-stream method
>>>>     and an
>>>>     RTP-single stream mixer method ( and also the method for
>>>>     multi-party
>>>>     unaware endpoints ). Then we decided to move on without the
>>>>     multi-stream
>>>>     method. It is a bit inconsistent to bring it in again, but I
>>>>     can see the
>>>>     possibility to arrange for E2E encryption to be a good reason. The
>>>>     earlier multi-stream method was based on the RTP-translator and
>>>>     required
>>>>     the mixer to recover from packet loss and insertion of marks for
>>>>     unrecoverable loss, and therefore could not be used for E2E
>>>>     encryption.
>>>>     Therefore it might be valid to rethink the earlier decision.
>>>>
>>>>     Questions:
>>>>
>>>>     1. Is 3GPP TS 26.114 Annex T an example of sdp for the kind of SFU
>>>>     described in RFC 7667 section 3.7, with one media section per
>>>>     stream in
>>>>     a lower number of streams than participants? If not, how does a
>>>>     normal
>>>>     sdp look for an SFU?
>>>>
>>>>     2. RFC 7667 section 3.7 seems to tell that the sequence number is
>>>>     regenerated in sequence by the middlebox. Is that right? That
>>>>     would make
>>>>     it not possible to detect if total loss of text occurred.
>>>>     Recovery can
>>>>     be done based on timestamp analysis, but not detection of
>>>>     unrecoverable
>>>>     loss.
>>>>
>>>>     3. How is the source conveyed? There is apparently an SSRC to SSRC
>>>>     mapping taking place. Is there a lower number of SSRCs used in
>>>>     transmission from the middlebox than the total number of
>>>>     participants?
>>>>     If so, how is switch of source indicated? By RTCP?
>>>>
>>>>
>>>>     If we find that it is likely possible to use the SFU, then I would
>>>>     anyway think that we just include general hints of its use in this
>>>>     draft, and let further work detail it. Would that be an acceptable
>>>>     conclusion?
>>>>
>>>>
>>>>     Regards
>>>>
>>>>     Gunnar
>>>>
>>>>     -- 
>>>>
>>>>     Gunnar Hellström
>>>>     GHAccess
>>>>     [e-mail] <mailto:[e-mail]>
>>>>
>>> -- 
>>> Gunnar Hellström
>>> GHAccess
>>> [e-mail]
>> -- 
>> Gunnar Hellström
>> GHAccess
>> [e-mail]
> -- 
> Gunnar Hellström
> GHAccess
> [e-mail]

-- 
Gunnar Hellström
GHAccess
[e-mail]