Re: [Rum] Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

[Paul Kyzivat <[e-mail]>]

I question how well RTT text is suited to multiparty conferences.

If you have messages on your screen from multiple parties, and many of 
them are updating in real-time, are you going to be able to perceive 
what is going on?

And while you can have a column per person for two-party and maybe 
3-party conversations, that doesn't scale up. With many parties, some 
typing may scroll off the screen before it is complete.

Perhaps for conferences it is better to just use line-at-a-time chat. If 
necessary, I presume there could be gateways between RTT and chat. A RUE 
could have the capability to negotiate down from RTT to chat.

	Thanks,
	Paul

On 8/28/19 2:15 AM, Gunnar Hellström wrote:
> Den 2019-08-27 kl. 23:15, skrev Brian Rosen:
>> At least by centralizing the problem at a “mixer”, Alice and Bob will 
>> see the same thing.
>>
>> You don’t have the problem in Instant Messaging, because you can’t 
>> backspace or delete a sent message.  Of course if multiple people are 
>> typing simultaneously in such systems, message order will be confusing 
>> in that instant.
> Right, it is a similar kind of problem that text appears in an 
> unexpected order. There is also at least one instant messaging service 
> that allows modification in already sent message. But I think it has 
> limitations to only accept that in the last message sent. it is 
> convenient anyway.
>>
>> Anyway, we need to specify the mixer for RTT so it receives each of 
>> the RTT streams and produces a single composite stream for each 
>> participant.
> 
> Yes, right, and there is an effort in that direction in:
> 
> http://www.realtimetext.org/sites/default/files/Files_and_Documents/Specifications/multiparty-real-time-text-mixer-2011-04-30.pdf
> 
> It is written for conference-unaware user devices.
> 
> The goals are specified as follows:
> 
> The procedures are intended to make best efforts to present a 
> multi-party text conversation on a terminal that has no awareness of 
> multi-party calls. There are some obvious drawbacks, and a terminal 
> designed with multi-party awareness will be able to present multi-party 
> call contents in a more flexible way. Only two parties at a time will be 
> allowed to display added text in real-time, while the other parties’ 
> produced text will need to be stored in the multi-party server for a 
> moment awaiting a suitable occasion to be displayed. There are also some 
> cases of erasure that will not be performed on the target text but only 
> indicated in another way. Even with these drawbacks, the procedure 
> provides an opportunity to display text from more than two parties in a 
> smooth and readable way.
> 
> ---------------------------------------------------------------------------------------------------
> 
> I see such mixer procedures as a fall-back for cases without conference 
> awareness, but want to see support for conference-aware terminals, where 
> text from more than two parties can be presented in real-time, and the 
> end user or app can have influence over the presentation style - e.g. 
> select between the multiple column view and the one-column-with-labels 
> view.  A mixer for that case would only need to assure that the receiver 
> has the right kind of multi-party awareness and send RTT text with 
> source information attached, and let the receiving terminal sort out the 
> presentation. This is already possible with CSRC and CNAME when using 
> RTP, but we lose that possibility natively when using the WebRTC data 
> channel to transport RTT, and would need to specify a way to include the 
> source also for that case.
> 
> ------------------------------------------------------
> 
> By the way, what is your current view of how to transport RTT for RUM, 
> now when you say that you will use WebRTC transports for media?
> 
> Regards
> 
> Gunnar
> 
>>
>>> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström 
>>> <[e-mail] <mailto:[e-mail]>> wrote:
>>>
>>> Den 2019-08-27 kl. 21:48, skrev Brian Rosen:
>>>> The problem of conference 4103 RTT is high on my list of work I need 
>>>> to get done.  So, I’m motivated to help out.
>>> Thanks, great.
>>>> The basic problem is that we’re going to get very inconsistent UI 
>>>> doing it that way, because of how systems will handle backspace of 
>>>> one party that extends beyond responses from other parties:
>>> (well, for me the currently most basic problem is to have a reliable 
>>> way to append received text to the already presented text of the 
>>> right participant. And that is getting worse in WebRTC than it was in 
>>> RFC 4103. But we will sort it out.)
>>>>
>>>> Alice: I waited for you
>>>> Bob: I didn’t see you
>>>> Alice: sorry
>>>>
>>>> And then Alice types 12 backspaces.
>>>>
>>>> What should happen?
>>>
>>> You are right that there are a number of ways to handle the RTT UI. 
>>> And just as inconsistencies are common with a message oriented UI, 
>>> where messages show up in a confusing order because two users 
>>> completed messages in an unexpected time order, it is possible that 
>>> RTT text gets displayed in a strange order after erasure and 
>>> retyping. It is better for RTT than for message oriented 
>>> presentation, and user get used to it in both cases.  With the 
>>> labelled style in one column you have in the example, I would 
>>> recommend that first 5 backspaces erase "sorry", next backspace 
>>> erases the line separator, and pulls down "I waited for you" to be 
>>> shown last, as an uncompleted text. Then the next 6 backspaces erase 
>>> so that only "I waited f" is displayed. When Alice adds text and end 
>>> with a new line, the corrected sentence is allowed to flow up when 
>>> new text is added from any participant.  That causes a bit strange 
>>> order, but it is just as manageable as when text in messaging 
>>> applications appear in an unexpected order so that one message seems 
>>> to be a respone on something totally else than what was intended.
>>>
>>> A sophisticated UI may mark text that is moved and modified.
>>>
>>> We want to keep sentences or at least phrases from each participant 
>>> together in a readable unit. Already that causes a design decision on 
>>> where to place the completed chunk of text once the user has 
>>> completed it. The start of the chunk may be older than completed text 
>>> from other participants which would motivate to move it up a bit in 
>>> the presentation. But the end of it is at that moment the latest text 
>>> to present. I think it is best to let the finished text be presented 
>>> last on the display, but let others' newer text push everything up 
>>> and be displayed last.
>>>
>>>
>>> T.140 has information on how to handle erasure:
>>>
>>> -------------------From T.140---------------------------
>>>
>>> 8.2 Erase last character
>>> Purpose: Erase the last character sent from the display at the 
>>> receiving end.
>>> Code: BS: 0008.
>>> Procedure: On the receiving end: Move the insertion point to the last 
>>> character and erase it.
>>> Combined characters are erased as a unit, with one BS erasing the 
>>> whole character even if it is
>>> combined from more than one component.
>>> Control sequences (like CR LF) are erased in one operation.
>>> NOTE – The same action shall be taken on the local display.
>>>
>>> ------------------------------------------------------------
>>>
>>>   /Gunnar
>>>
>>>>
>>>> Brian
>>>>
>>>>> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström 
>>>>> <[e-mail] <mailto:[e-mail]>> 
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> A topic is currently discussed in mmusic that is closely related to 
>>>>> rum. it is WebRTC transport of real-time text.
>>>>>
>>>>> The draft is draft-holmberg-mmusic-t140-usage-data-channel .
>>>>>
>>>>> A good point to start reading could be:
>>>>>
>>>>> https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-holmberg-mmusic-t140-usage-data-channel
>>>>>
>>>>> Please check if the current state of the discussion suits rum!
>>>>>
>>>>> The only issue that seems to be remaining is how to transport RTT 
>>>>> data to and from a conference server that combines all traffic per 
>>>>> media in a meeting in one data stream. That is not very elegantly 
>>>>> specified for RFC 4103 transport of RTT in RTP either, so we might 
>>>>> want to do a rapid action together to solve the multi-party RTT MCU 
>>>>> case in a general and consistent way.
>>>>>
>>>>> Regards
>>>>>
>>>>> Gunnar
>>>>>
>>>>> --
>>>>> -----------------------------------------
>>>>> Gunnar Hellström
>>>>> Omnitor
>>>>> [e-mail]
>>>>> [phone]
>>>>>
>>>>>
>>> --
>>> -----------------------------------------
>>> Gunnar Hellström
>>> Omnitor
>>> [e-mail] <mailto:[e-mail]>
>>> [phone]
>>
>>
> -- 
> -----------------------------------------
> Gunnar Hellström
> Omnitor
> [e-mail]
> [phone]
> 
>