Re: [MMUSIC] draft-holmberg-mmusic-t140-usage-data-channel - multi-party

Gunnar Hellström <gunnar.hellstrom@omnitor.se> Thu, 29 August 2019 10:50 UTC

Return-Path: <gunnar.hellstrom@omnitor.se>
X-Original-To: mmusic@ietfa.amsl.com
Delivered-To: mmusic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 44A41120813 for <mmusic@ietfa.amsl.com>; Thu, 29 Aug 2019 03:50:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2v5lBNk1d4dT for <mmusic@ietfa.amsl.com>; Thu, 29 Aug 2019 03:50:45 -0700 (PDT)
Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E3B641201CE for <mmusic@ietf.org>; Thu, 29 Aug 2019 03:50:44 -0700 (PDT)
X-Halon-ID: d6caf30b-ca4a-11e9-903a-005056917f90
Authorized-sender: gunnar.hellstrom@omnitor.se
Received: from [192.168.2.136] (unknown [88.129.173.120]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPSA id d6caf30b-ca4a-11e9-903a-005056917f90; Thu, 29 Aug 2019 12:50:39 +0200 (CEST)
To: Christer Holmberg <christer.holmberg@ericsson.com>, "Roni Even (A)" <roni.even@huawei.com>, "mmusic@ietf.org" <mmusic@ietf.org>
References: <49749CEF-41E8-4E87-8CC6-938DBDA0CEE7@ericsson.com> <2201665d-5054-1872-d208-a0fe2d26095c@omnitor.se> <VI1PR07MB3167055C995D17D4BA9E36DE93A50@VI1PR07MB3167.eurprd07.prod.outlook.com> <8d14b055-8405-4a4f-174d-d7580bea348c@omnitor.se> <0DA1248C-41FC-4155-A578-29A19883857C@ericsson.com> <a91850b9-6e86-058f-dddd-3f856bcd6710@omnitor.se> <DBC532B8-38DC-4140-B7C4-0B6853F0EF77@ericsson.com> <6fcf46a6-544d-027c-97c7-5c0e08caa555@omnitor.se> <HE1PR07MB3161A9A0C696B9636BBD380C93A70@HE1PR07MB3161.eurprd07.prod.outlook.com> <6484f305-0c38-4178-ee12-05a7dc38364f@omnitor.se> <6E58094ECC8D8344914996DAD28F1CCD23D43936@dggemm526-mbx.china.huawei.com> <65CFD7ED-04EA-498C-9CB3-18F49C04AF73@ericsson.com> <6E58094ECC8D8344914996DAD28F1CCD23D43990@dggemm526-mbx.china.huawei.com> <5D9A5640-FCF5-4888-9B05-58EE0468511F@ericsson.com> <6E58094ECC8D8344914996DAD28F1CCD23D439CA@dggemm526-mbx.china.huawei.com> <F6C8AB24-0303-43C2-A4D6-2161000457F7@ericsson.com>
From: =?UTF-8?Q?Gunnar_Hellstr=c3=b6m?= <gunnar.hellstrom@omnitor.se>
Message-ID: <e93615fd-af6b-5a46-e69f-f0004822a1a2@omnitor.se>
Date: Thu, 29 Aug 2019 12:50:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <F6C8AB24-0303-43C2-A4D6-2161000457F7@ericsson.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/mmusic/UueB24peeff0_xKMwUYnlsn1FGk>
Subject: Re: [MMUSIC] draft-holmberg-mmusic-t140-usage-data-channel - multi-party
X-BeenThere: mmusic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Multiparty Multimedia Session Control Working Group <mmusic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mmusic>, <mailto:mmusic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mmusic/>
List-Post: <mailto:mmusic@ietf.org>
List-Help: <mailto:mmusic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mmusic>, <mailto:mmusic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Aug 2019 10:50:49 -0000

Hi,

We need to remember also Paul's question, if we really need a 
multi-party case with a central server combining real-time text from 
many sources in one data channel and have source indication on every 
T.140 element so that the receiving client can make its own presentation 
layout of the real-time text conference.

Maybe we can assume that for the case that the receiving client want to 
make its own layout, the case with one T.140 data channel per active 
user is realistic and sufficient.

The case with the central server combining a number of real-time text 
sources into one T.140 data channel could be reserved for the case when 
a T.140 mixer dictates layout, and places labels in IRC style in front 
of text from the participants and makes the best of trying to present 
real-time text in that way. It will not be nice, but still possible with 
some limitations. It is similar to a video mixer dictating the layout of 
a number of participants in one combined view, and to audio mixing, 
allowing audio from a number of participants be heard in the same 
channel with no possibility for the receiving user to influence mixing. 
Maybe it is just fair to have that kind of fixed mixing for real-time 
text when the other media have it.

I have always had a view that it was more important to separate the 
sources for real-time text and let the receiving party do its own layout 
than it would be for video and audio participants to do such separation 
on video and audio. But I might be wrong.

See also inline

Den 2019-08-29 kl. 11:50, skrev Christer Holmberg:
> Hi,
>
>>>> I you want to multiplex but know who is contributing the data you need
>>>> some identifier as part of the T.140 data packet similar to SSRC in the RTP header.
>>>     
>>>   Exactly, and that is what have been discussed :)
>>>     
>>>> It is not even similar to CSRC since CSRC is more about providing
>>>> about the content of this specific RTP packet but in this case each t.140 packet is
>>>> coming from a single source so one identifier is needed.
If it was from a combining server on the RTP side, I think it would be 
CSRC. The SSRC would then point at the server.
>>>     
>>> Your comment above shows why I have suggested to take the discussion to AVT: there you will find
>>>   people (like yourself) that know more about that kind of stuff :)
>>     
>>   RE: This is not AVT and not MMUSIC since it is some encapsulation for T.140 which is not RTP.
>> This is a topic for dispatch since it may be a single draft that needs a home
> We are using the RFC 4103 T140block also on the data channel, so *IF* the solution would be an update to that, wouldn't it belong to AVT?
>
> *IF* the solution is an update to T.140, I guess that should be done in ITU-T?

T140blocks are just a number of complete T.140 code elements. And it is 
possible to define new such code elements.

So, even if RFC 4103 defined the term T140block, it is in ITU-T that 
additions to it should be defined.

If instead we specify that the T.140 data channel  shall carry a source 
and a T.140block in each T.140 data channel message, then it is a matter 
for mmusic and the draft we discuss here (or AVT or the closed RTCWEB if 
it seems complex) to specify that source element and the structure of 
the T.140 data channel message contents.

Regards

Gunnar

>
> Having said that, before going into solutions there should be *A* venue to discuss the problem.
>
> Regards,
>
> Christer
>      
>      
>      
>       
>       
>      From: Christer Holmberg [mailto:christer.holmberg@ericsson.com]
>      Sent: Thursday, August 29, 2019 12:08 PM
>      To: Roni Even (A); Gunnar Hellström; mmusic@ietf.org
>      Subject: Re: [MMUSIC] draft-holmberg-mmusic-t140-usage-data-channel - multi-party
>       
>      Hi Roni,
>       
>      >My view is that is we are talking about WEBRTC, the multiparty and user information will be part of the communication between the application >server and the browser and the application server is doing whatever mixing or forwarding that it needs.
>      >I can see that there must be some way for the application server to know what is each t.140 data channel   if there is more than one , some >distinction is the language attribute. If you want more maybe as Christer suggested use the “content” attribute whose objective is to provide >information about stream.
>       
>      The main issue is if you want to use a *single* data channel for T.140 traffic from multiple remote participants. In that case you need to be able to indicate the source of the T.140 data, but unlike RTP you don’t have SSRC etc for doing that.
>       
>      Regards,
>       
>      Christer
>       
>       
>      From: mmusic [mailto:mmusic-bounces@ietf.org] On Behalf Of Gunnar Hellström
>      Sent: Sunday, August 25, 2019 9:49 AM
>      To: Christer Holmberg; mailto:mmusic@ietf.org
>      Subject: Re: [MMUSIC] draft-holmberg-mmusic-t140-usage-data-channel - multi-party
>       
>      Hi Christer,
>      Below are comments on the multi-party considerations
>       
>      Den 2019-08-24 kl. 12:04, skrev Christer Holmberg:
>      --------------------------------------------------------------------------------------------------------------------------------------------
>       
>      18. Add a new section 4.5
>       
>      4.5 Multi-party considerations
>       
>      Implementations should be prepared to accept establishment and use of multiple T140 data channels in order to support multi-party sessions with real-time text. A number of scenarios are available for how multi-party sessions can be supported in the WebRTC environment.
>      Implementations may select any suitable scenario.
>      I don't think we need the two last sentences.
>       
>      Also, in some cases all communication will go via a central server, so there will only be one T.140 data channel towards each participant.
>      No, T.140 has no source indicator of its own, it relies on the transport to indicate the source for each T140block. In RTP, this can be done by an RTP mixer making one stream from multiple sources including CSRC for the sources of the primary text and for the redundant generations of text in each packet. On the T140 data channel side, I do not know any corresponding way to indicate different sources in the same data channel.
>      The solutions I see are: 1) create one T140 channel per source/destination pair. or 2) Introduce a source indicator in the data format for the T140 data channel, either one per STCP message requiring all T140blocks in the message being from one source only, or inline between series of T140blocks from different sources. This is because the real-time text from multiple sources simultaneously need to be presented with some separation, so that the text gets readable at least sentence-wise from each source. The T.140 Appendix 1 shows two ways to do this, one column-oriented, and one sentence-oriented with a label per start of sentence. You can read more about the topic in https://www.ietf.org/archive/id/draft-hellstrom-text-conference-04.txt
>       
>       
>      So, maybe something like:
>       
>      "In order for an implementation to be able to support multi-party scenarios where each participant will communicate directly
>      with the other participants, the implementation need to be able to support multiple simultaneous T.140 data channels."
>      While that is true, it does not tell us how to solve the case with a conference server.
>      
>      
>       
>       
>      Presentation should be made so that the source of the real-time text is perceivable and the relative time relations in the conversation approximately presented.
>      The "label" attribute may be used to convey a presentable source.
>      I am not sure I understand the "relative time relations" part.
>      In order to enable the reader to follow the flow of a multi-party text conversation, it is a good habit to present older text placed higher in the text area and newer text placed lower. (This is valid for both when you present text in one column per source and if you combine all sources in one (IRC-style) column).
>      It is also a good habit to present text from the same source readable together, e.g. sentence by sentence, (and not break the text just because a text item from another source was received during the time the sentence was created).
>      These two requirements are in conflict. A true time-related presentation would fragment simultaneous text from different sources into unreadability, and presenting all text from each source in one chunk each would give no clue about the flow of the discussion.
>      Therefore this expression " the relative time relations in the conversation approximately presented".
>       
>       
>      Regarding the source, perhaps extending my suggested text above with something like:
>       
>      "In order for an implementation to be able to support multi-party scenarios where each participant will communicate directly
>      with the other participants, the implementation need to be able to support multiple simultaneous T.140 data channels. The label
>      attribute can be used to provide information that helps an implementation to distinguish between the T.140 data channels."
>       
>      Yes, this is a good statement for the case without the server, or can be modified for a server that maintains a channel per source. But which solution do you prefer if we allow a mixing server?
>      1) require also servers to support one T140 data channel per source
>      2) introduce a data format for the T140 data channel containing a unique source identifier
>      3) introduce a source identifier in-line in the T.140 data stream. (T.140 is extendable)
>       
>      /Gunnar
>      ----------------------------------------------------------------------------------------------------------------------------------
>       
>      -
>      -
>      -----------------------------------------
>      Gunnar Hellström
>      Omnitor
>      mailto:gunnar.hellstrom@omnitor.se
>      +46 708 204 288
>      
>      
>
> _______________________________________________
> mmusic mailing list
> mmusic@ietf.org
> https://www.ietf.org/mailman/listinfo/mmusic

-- 
-----------------------------------------
Gunnar Hellström
Omnitor
gunnar.hellstrom@omnitor.se
+46 708 204 288