Re: [Rum] [EXT] Re: Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

[Paul Kyzivat <[e-mail]>]

On 8/29/19 1:17 PM, Brian Rosen wrote:
> I think we want a rum device to be able to be a good conference 
> participant, but like nearly all conference participants, they only see 
> a 2 party call and the bridge does the magic.
> I don’t think we need to require anything else.  I think we want a 
> bridge to be able to host a rum-compliant endpoint.

I agree conceptually. But we need to recognize that we are out ahead of 
VRS. Near term there is no way to get an audio/video/RTT bridge into a 
call with a VRS user. Because the bridge is not in the VRS iTRS it will 
only be able to get connected as an audio call with and interpreter.

The first form of conference is likely to be for emergency calls. But 
there isn't yet a specification of how to connect VRS to ng911 to get an 
audio/video/RTT emergency call.

The other possibility is via Direct Video Calling using the VATRP system 
from Mitre. I guess in theory that is possible now. But it will 
currently have VRS users that use VRS Provider proprietary devices. 
Anything to support conferencing that requires extra signaling won't 
work until VRS users can actually use RUM-compliant RUEs. But that could 
work with vanilla RTT as long as the DVC device sets up the conferencing.

	Thanks,
	Paul

> Brian
> 
>> On Aug 29, 2019, at 12:20 PM, Malloy, Jim <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Do we intend for the RUE to accommodate multiple party calls (more 
>> than three)? I don't recall any language related to anything but a 
>> point to point call.  For emergency calls, conferencing in the PSAP 
>> (three parties) may be appropriate.  Are we looking for more than that?
>>
>> --Jim Malloy
>>
>> -----Original Message-----
>> From: Rum <[e-mail] <mailto:[e-mail]>> On 
>> Behalf Of Paul Kyzivat
>> Sent: Thursday, August 29, 2019 12:05 PM
>> To:[e-mail] <mailto:[e-mail]>
>> Subject: [EXT] Re: [Rum] Real-time text in WebRTC is discussed in 
>> mmusic - a topic closely telated to rum
>>
>> I question how well RTT text is suited to multiparty conferences.
>>
>> If you have messages on your screen from multiple parties, and many of 
>> them are updating in real-time, are you going to be able to perceive 
>> what is going on?
>>
>> And while you can have a column per person for two-party and maybe 
>> 3-party conversations, that doesn't scale up. With many parties, some 
>> typing may scroll off the screen before it is complete.
>>
>> Perhaps for conferences it is better to just use line-at-a-time chat. 
>> If necessary, I presume there could be gateways between RTT and chat. 
>> A RUE could have the capability to negotiate down from RTT to chat.
>>
>> Thanks,
>> Paul
>>
>> On 8/28/19 2:15 AM, Gunnar Hellström wrote:
>>> Den 2019-08-27 kl. 23:15, skrev Brian Rosen:
>>>> At least by centralizing the problem at a “mixer”, Alice and Bob will
>>>> see the same thing.
>>>>
>>>> You don’t have the problem in Instant Messaging, because you can’t
>>>> backspace or delete a sent message.  Of course if multiple people are
>>>> typing simultaneously in such systems, message order will be
>>>> confusing in that instant.
>>> Right, it is a similar kind of problem that text appears in an
>>> unexpected order. There is also at least one instant messaging service
>>> that allows modification in already sent message. But I think it has
>>> limitations to only accept that in the last message sent. it is
>>> convenient anyway.
>>>>
>>>> Anyway, we need to specify the mixer for RTT so it receives each of
>>>> the RTT streams and produces a single composite stream for each
>>>> participant.
>>>
>>> Yes, right, and there is an effort in that direction in:
>>>
>>> http://www.realtimetext.org/sites/default/files/Files_and_Documents/Sp
>>> ecifications/multiparty-real-time-text-mixer-2011-04-30.pdf
>>>
>>> It is written for conference-unaware user devices.
>>>
>>> The goals are specified as follows:
>>>
>>> The procedures are intended to make best efforts to present a
>>> multi-party text conversation on a terminal that has no awareness of
>>> multi-party calls. There are some obvious drawbacks, and a terminal
>>> designed with multi-party awareness will be able to present
>>> multi-party call contents in a more flexible way. Only two parties at
>>> a time will be allowed to display added text in real-time, while the 
>>> other parties’
>>> produced text will need to be stored in the multi-party server for a
>>> moment awaiting a suitable occasion to be displayed. There are also
>>> some cases of erasure that will not be performed on the target text
>>> but only indicated in another way. Even with these drawbacks, the
>>> procedure provides an opportunity to display text from more than two
>>> parties in a smooth and readable way.
>>>
>>> ----------------------------------------------------------------------
>>> -----------------------------
>>>
>>> I see such mixer procedures as a fall-back for cases without
>>> conference awareness, but want to see support for conference-aware
>>> terminals, where text from more than two parties can be presented in
>>> real-time, and the end user or app can have influence over the 
>>> presentation style - e.g.
>>> select between the multiple column view and the one-column-with-labels
>>> view.  A mixer for that case would only need to assure that the
>>> receiver has the right kind of multi-party awareness and send RTT text
>>> with source information attached, and let the receiving terminal sort
>>> out the presentation. This is already possible with CSRC and CNAME
>>> when using RTP, but we lose that possibility natively when using the
>>> WebRTC data channel to transport RTT, and would need to specify a way
>>> to include the source also for that case.
>>>
>>> ------------------------------------------------------
>>>
>>> By the way, what is your current view of how to transport RTT for RUM,
>>> now when you say that you will use WebRTC transports for media?
>>>
>>> Regards
>>>
>>> Gunnar
>>>
>>>>
>>>>> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström
>>>>> <[e-mail] 
>>>>> <mailto:[e-mail]><mailto:[e-mail]>> 
>>>>> wrote:
>>>>>
>>>>> Den 2019-08-27 kl. 21:48, skrev Brian Rosen:
>>>>>> The problem of conference 4103 RTT is high on my list of work I
>>>>>> need to get done.  So, I’m motivated to help out.
>>>>> Thanks, great.
>>>>>> The basic problem is that we’re going to get very inconsistent UI
>>>>>> doing it that way, because of how systems will handle backspace of
>>>>>> one party that extends beyond responses from other parties:
>>>>> (well, for me the currently most basic problem is to have a reliable
>>>>> way to append received text to the already presented text of the
>>>>> right participant. And that is getting worse in WebRTC than it was
>>>>> in RFC 4103. But we will sort it out.)
>>>>>>
>>>>>> Alice: I waited for you
>>>>>> Bob: I didn’t see you
>>>>>> Alice: sorry
>>>>>>
>>>>>> And then Alice types 12 backspaces.
>>>>>>
>>>>>> What should happen?
>>>>>
>>>>> You are right that there are a number of ways to handle the RTT UI.
>>>>> And just as inconsistencies are common with a message oriented UI,
>>>>> where messages show up in a confusing order because two users
>>>>> completed messages in an unexpected time order, it is possible that
>>>>> RTT text gets displayed in a strange order after erasure and
>>>>> retyping. It is better for RTT than for message oriented
>>>>> presentation, and user get used to it in both cases.  With the
>>>>> labelled style in one column you have in the example, I would
>>>>> recommend that first 5 backspaces erase "sorry", next backspace
>>>>> erases the line separator, and pulls down "I waited for you" to be
>>>>> shown last, as an uncompleted text. Then the next 6 backspaces erase
>>>>> so that only "I waited f" is displayed. When Alice adds text and end
>>>>> with a new line, the corrected sentence is allowed to flow up when
>>>>> new text is added from any participant.  That causes a bit strange
>>>>> order, but it is just as manageable as when text in messaging
>>>>> applications appear in an unexpected order so that one message seems
>>>>> to be a respone on something totally else than what was intended.
>>>>>
>>>>> A sophisticated UI may mark text that is moved and modified.
>>>>>
>>>>> We want to keep sentences or at least phrases from each participant
>>>>> together in a readable unit. Already that causes a design decision
>>>>> on where to place the completed chunk of text once the user has
>>>>> completed it. The start of the chunk may be older than completed
>>>>> text from other participants which would motivate to move it up a
>>>>> bit in the presentation. But the end of it is at that moment the
>>>>> latest text to present. I think it is best to let the finished text
>>>>> be presented last on the display, but let others' newer text push
>>>>> everything up and be displayed last.
>>>>>
>>>>>
>>>>> T.140 has information on how to handle erasure:
>>>>>
>>>>> -------------------From T.140---------------------------
>>>>>
>>>>> 8.2 Erase last character
>>>>> Purpose: Erase the last character sent from the display at the
>>>>> receiving end.
>>>>> Code: BS: 0008.
>>>>> Procedure: On the receiving end: Move the insertion point to the
>>>>> last character and erase it.
>>>>> Combined characters are erased as a unit, with one BS erasing the
>>>>> whole character even if it is combined from more than one component.
>>>>> Control sequences (like CR LF) are erased in one operation.
>>>>> NOTE – The same action shall be taken on the local display.
>>>>>
>>>>> ------------------------------------------------------------
>>>>>
>>>>> /Gunnar
>>>>>
>>>>>>
>>>>>> Brian
>>>>>>
>>>>>>> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström
>>>>>>> <[e-mail] 
>>>>>>> <mailto:[e-mail]><mailto:[e-mail]>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> A topic is currently discussed in mmusic that is closely related
>>>>>>> to rum. it is WebRTC transport of real-time text.
>>>>>>>
>>>>>>> The draft is draft-holmberg-mmusic-t140-usage-data-channel .
>>>>>>>
>>>>>>> A good point to start reading could be:
>>>>>>>
>>>>>>> https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-hol
>>>>>>> mberg-mmusic-t140-usage-data-channel
>>>>>>>
>>>>>>> Please check if the current state of the discussion suits rum!
>>>>>>>
>>>>>>> The only issue that seems to be remaining is how to transport RTT
>>>>>>> data to and from a conference server that combines all traffic per
>>>>>>> media in a meeting in one data stream. That is not very elegantly
>>>>>>> specified for RFC 4103 transport of RTT in RTP either, so we might
>>>>>>> want to do a rapid action together to solve the multi-party RTT
>>>>>>> MCU case in a general and consistent way.
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Gunnar
>>>>>>>
>>>>>>> --
>>>>>>> -----------------------------------------
>>>>>>> Gunnar Hellström
>>>>>>> Omnitor
>>>>>>> [e-mail] <mailto:[e-mail]>
>>>>>>> [phone]
>>>>>>>
>>>>>>>
>>>>> --
>>>>> -----------------------------------------
>>>>> Gunnar Hellström
>>>>> Omnitor
>>>>> [e-mail] 
>>>>> <mailto:[e-mail]><mailto:[e-mail]>
>>>>> [phone]
>>>>
>>>>
>>> --
>>> -----------------------------------------
>>> Gunnar Hellström
>>> Omnitor
>>> [e-mail] <mailto:[e-mail]>
>>> [phone]
>>>
>>>
>>
>> --
>> Rum mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/rum
>> --
>> Rum mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/rum
>