Re: [Rum] [EXT] Re: Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

[Brian Rosen <[e-mail]>]

I think we want a rum device to be able to be a good conference participant, but like nearly all conference participants, they only see a 2 party call and the bridge does the magic.
I don’t think we need to require anything else.  I think we want a bridge to be able to host a rum-compliant endpoint.

Brian

> On Aug 29, 2019, at 12:20 PM, Malloy, Jim <[e-mail]> wrote:
> 
> Do we intend for the RUE to accommodate multiple party calls (more than three)? I don't recall any language related to anything but a point to point call.  For emergency calls, conferencing in the PSAP (three parties) may be appropriate.  Are we looking for more than that?
> 
> --Jim Malloy
> 
> -----Original Message-----
> From: Rum <[e-mail] <mailto:[e-mail]>> On Behalf Of Paul Kyzivat
> Sent: Thursday, August 29, 2019 12:05 PM
> To: [e-mail] <mailto:[e-mail]>
> Subject: [EXT] Re: [Rum] Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum
> 
> I question how well RTT text is suited to multiparty conferences.
> 
> If you have messages on your screen from multiple parties, and many of them are updating in real-time, are you going to be able to perceive what is going on?
> 
> And while you can have a column per person for two-party and maybe 3-party conversations, that doesn't scale up. With many parties, some typing may scroll off the screen before it is complete.
> 
> Perhaps for conferences it is better to just use line-at-a-time chat. If necessary, I presume there could be gateways between RTT and chat. A RUE could have the capability to negotiate down from RTT to chat.
> 
> 	Thanks,
> 	Paul
> 
> On 8/28/19 2:15 AM, Gunnar Hellström wrote:
>> Den 2019-08-27 kl. 23:15, skrev Brian Rosen:
>>> At least by centralizing the problem at a “mixer”, Alice and Bob will 
>>> see the same thing.
>>> 
>>> You don’t have the problem in Instant Messaging, because you can’t 
>>> backspace or delete a sent message.  Of course if multiple people are 
>>> typing simultaneously in such systems, message order will be 
>>> confusing in that instant.
>> Right, it is a similar kind of problem that text appears in an 
>> unexpected order. There is also at least one instant messaging service 
>> that allows modification in already sent message. But I think it has 
>> limitations to only accept that in the last message sent. it is 
>> convenient anyway.
>>> 
>>> Anyway, we need to specify the mixer for RTT so it receives each of 
>>> the RTT streams and produces a single composite stream for each 
>>> participant.
>> 
>> Yes, right, and there is an effort in that direction in:
>> 
>> http://www.realtimetext.org/sites/default/files/Files_and_Documents/Sp
>> ecifications/multiparty-real-time-text-mixer-2011-04-30.pdf
>> 
>> It is written for conference-unaware user devices.
>> 
>> The goals are specified as follows:
>> 
>> The procedures are intended to make best efforts to present a 
>> multi-party text conversation on a terminal that has no awareness of 
>> multi-party calls. There are some obvious drawbacks, and a terminal 
>> designed with multi-party awareness will be able to present 
>> multi-party call contents in a more flexible way. Only two parties at 
>> a time will be allowed to display added text in real-time, while the other parties’
>> produced text will need to be stored in the multi-party server for a 
>> moment awaiting a suitable occasion to be displayed. There are also 
>> some cases of erasure that will not be performed on the target text 
>> but only indicated in another way. Even with these drawbacks, the 
>> procedure provides an opportunity to display text from more than two 
>> parties in a smooth and readable way.
>> 
>> ----------------------------------------------------------------------
>> -----------------------------
>> 
>> I see such mixer procedures as a fall-back for cases without 
>> conference awareness, but want to see support for conference-aware 
>> terminals, where text from more than two parties can be presented in 
>> real-time, and the end user or app can have influence over the presentation style - e.g.
>> select between the multiple column view and the one-column-with-labels 
>> view.  A mixer for that case would only need to assure that the 
>> receiver has the right kind of multi-party awareness and send RTT text 
>> with source information attached, and let the receiving terminal sort 
>> out the presentation. This is already possible with CSRC and CNAME 
>> when using RTP, but we lose that possibility natively when using the 
>> WebRTC data channel to transport RTT, and would need to specify a way 
>> to include the source also for that case.
>> 
>> ------------------------------------------------------
>> 
>> By the way, what is your current view of how to transport RTT for RUM, 
>> now when you say that you will use WebRTC transports for media?
>> 
>> Regards
>> 
>> Gunnar
>> 
>>> 
>>>> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström 
>>>> <[e-mail] <mailto:[e-mail]> <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
>>>> 
>>>> Den 2019-08-27 kl. 21:48, skrev Brian Rosen:
>>>>> The problem of conference 4103 RTT is high on my list of work I 
>>>>> need to get done.  So, I’m motivated to help out.
>>>> Thanks, great.
>>>>> The basic problem is that we’re going to get very inconsistent UI 
>>>>> doing it that way, because of how systems will handle backspace of 
>>>>> one party that extends beyond responses from other parties:
>>>> (well, for me the currently most basic problem is to have a reliable 
>>>> way to append received text to the already presented text of the 
>>>> right participant. And that is getting worse in WebRTC than it was 
>>>> in RFC 4103. But we will sort it out.)
>>>>> 
>>>>> Alice: I waited for you
>>>>> Bob: I didn’t see you
>>>>> Alice: sorry
>>>>> 
>>>>> And then Alice types 12 backspaces.
>>>>> 
>>>>> What should happen?
>>>> 
>>>> You are right that there are a number of ways to handle the RTT UI. 
>>>> And just as inconsistencies are common with a message oriented UI, 
>>>> where messages show up in a confusing order because two users 
>>>> completed messages in an unexpected time order, it is possible that 
>>>> RTT text gets displayed in a strange order after erasure and 
>>>> retyping. It is better for RTT than for message oriented 
>>>> presentation, and user get used to it in both cases.  With the 
>>>> labelled style in one column you have in the example, I would 
>>>> recommend that first 5 backspaces erase "sorry", next backspace 
>>>> erases the line separator, and pulls down "I waited for you" to be 
>>>> shown last, as an uncompleted text. Then the next 6 backspaces erase 
>>>> so that only "I waited f" is displayed. When Alice adds text and end 
>>>> with a new line, the corrected sentence is allowed to flow up when 
>>>> new text is added from any participant.  That causes a bit strange 
>>>> order, but it is just as manageable as when text in messaging 
>>>> applications appear in an unexpected order so that one message seems 
>>>> to be a respone on something totally else than what was intended.
>>>> 
>>>> A sophisticated UI may mark text that is moved and modified.
>>>> 
>>>> We want to keep sentences or at least phrases from each participant 
>>>> together in a readable unit. Already that causes a design decision 
>>>> on where to place the completed chunk of text once the user has 
>>>> completed it. The start of the chunk may be older than completed 
>>>> text from other participants which would motivate to move it up a 
>>>> bit in the presentation. But the end of it is at that moment the 
>>>> latest text to present. I think it is best to let the finished text 
>>>> be presented last on the display, but let others' newer text push 
>>>> everything up and be displayed last.
>>>> 
>>>> 
>>>> T.140 has information on how to handle erasure:
>>>> 
>>>> -------------------From T.140---------------------------
>>>> 
>>>> 8.2 Erase last character
>>>> Purpose: Erase the last character sent from the display at the 
>>>> receiving end.
>>>> Code: BS: 0008.
>>>> Procedure: On the receiving end: Move the insertion point to the 
>>>> last character and erase it.
>>>> Combined characters are erased as a unit, with one BS erasing the 
>>>> whole character even if it is combined from more than one component.
>>>> Control sequences (like CR LF) are erased in one operation.
>>>> NOTE – The same action shall be taken on the local display.
>>>> 
>>>> ------------------------------------------------------------
>>>> 
>>>>   /Gunnar
>>>> 
>>>>> 
>>>>> Brian
>>>>> 
>>>>>> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström 
>>>>>> <[e-mail] <mailto:[e-mail]> <mailto:[e-mail] <mailto:[e-mail]>>>
>>>>>> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> A topic is currently discussed in mmusic that is closely related 
>>>>>> to rum. it is WebRTC transport of real-time text.
>>>>>> 
>>>>>> The draft is draft-holmberg-mmusic-t140-usage-data-channel .
>>>>>> 
>>>>>> A good point to start reading could be:
>>>>>> 
>>>>>> https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-hol <https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-hol>
>>>>>> mberg-mmusic-t140-usage-data-channel
>>>>>> 
>>>>>> Please check if the current state of the discussion suits rum!
>>>>>> 
>>>>>> The only issue that seems to be remaining is how to transport RTT 
>>>>>> data to and from a conference server that combines all traffic per 
>>>>>> media in a meeting in one data stream. That is not very elegantly 
>>>>>> specified for RFC 4103 transport of RTT in RTP either, so we might 
>>>>>> want to do a rapid action together to solve the multi-party RTT 
>>>>>> MCU case in a general and consistent way.
>>>>>> 
>>>>>> Regards
>>>>>> 
>>>>>> Gunnar
>>>>>> 
>>>>>> --
>>>>>> -----------------------------------------
>>>>>> Gunnar Hellström
>>>>>> Omnitor
>>>>>> [e-mail] <mailto:[e-mail]>
>>>>>> [phone]
>>>>>> 
>>>>>> 
>>>> --
>>>> -----------------------------------------
>>>> Gunnar Hellström
>>>> Omnitor
>>>> [e-mail] <mailto:[e-mail]> <mailto:[e-mail] <mailto:[e-mail]>>
>>>> [phone]
>>> 
>>> 
>> --
>> -----------------------------------------
>> Gunnar Hellström
>> Omnitor
>> [e-mail] <mailto:[e-mail]>
>> [phone]
>> 
>> 
> 
> --
> Rum mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/rum <https://www.ietf.org/mailman/listinfo/rum>
> -- 
> Rum mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/rum <https://www.ietf.org/mailman/listinfo/rum>