Re: [MMUSIC] Open source SIP/WebRTC real-time text integration

Christer Holmberg <christer.holmberg@ericsson.com> Wed, 18 December 2019 14:54 UTC

Return-Path: <christer.holmberg@ericsson.com>
X-Original-To: mmusic@ietfa.amsl.com
Delivered-To: mmusic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4268C120220 for <mmusic@ietfa.amsl.com>; Wed, 18 Dec 2019 06:54:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.002
X-Spam-Level:
X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wgaXjcNO9q8l for <mmusic@ietfa.amsl.com>; Wed, 18 Dec 2019 06:54:45 -0800 (PST)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80084.outbound.protection.outlook.com [40.107.8.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFB4D1201EA for <mmusic@ietf.org>; Wed, 18 Dec 2019 06:54:44 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b2Y+mow8IvY+zmh4YoKmuMQDYjvYyXLI/+19DcjoSdf+t3w2WqpbKjm6tjl1vE+lHyrLFY3rE2WFdZrWgYVeGE4bkQVdPCcl5qdCvLrnr3w15NTHoxJHzF/l0WA0lHhRmSvfiLUIyzrSgIypCdhXVO4dfhA1Lplo75YZ+Ji2td2G/z4qminrcmAQLpjp1FRcTls6JPAffk8m9hAF9qf63Bqp9r9KQz+Cnql4nqV7mIBSQzgxkrM5Gs7cOnlJlzLYHqYPSiH4MLOKdIegfC1cPg5JMTmEEB4lusotfZEZxt2JqtmdLPUOxD66T6qa3URCzs8qdNes1TSiLc3XLtp85w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XJJVT1XFSq37LFZBtx5CJf2Bd4LRpMU5b2R5DZcrmzM=; b=iNxURzCrmtbrck75BYeVVi5ZfLPn8MOn5NEkk8Kuyy8taNnG6KiM8uSnf+fp0DFuSzG8bqJxyxqPJRxX9OMgoxKRXh2BDADCTbQBPMPuSofVfBsyF/NkDdgOVeJHAAu0o4sJ4QuAqyxBM/3n+bZkxFB1j9yqmaoUwoipEv760JsbiRXMAMQBGwdap/ZY7W7ZS3MyU/fgjZnx6kgNvEHT8DnWDX6Gqqt+VtwZ7IsAFgEze9N8C8t7xGB73yWZVEQ76G4y/OqhIPi8UBLv9ynQ/seyKFqee9QiTQil01bRiI7zWNS4nSds+VA77NTIn19wHb54DeZkUg53dBFK3ZxTDg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XJJVT1XFSq37LFZBtx5CJf2Bd4LRpMU5b2R5DZcrmzM=; b=DP6A7r1xUis+9ilOdNb5PmBgWv0jVJhkUupw73UyupJmGDYjSR/DT3L2Swep+8exyjLyqweNrwkCu+bLKC65ViwMHGVKFo964VyvybznMAOkUOlRiudhDcjWoBw3AJQkhCgzvSq7PMhU5aEZUNdQOIRRTMf0YH0W0rO/xTV0iA4=
Received: from DB6PR0701MB2421.eurprd07.prod.outlook.com (10.168.73.16) by DB6PR0701MB2600.eurprd07.prod.outlook.com (10.169.215.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.11; Wed, 18 Dec 2019 14:54:42 +0000
Received: from DB6PR0701MB2421.eurprd07.prod.outlook.com ([fe80::39bd:a590:dcd9:201e]) by DB6PR0701MB2421.eurprd07.prod.outlook.com ([fe80::39bd:a590:dcd9:201e%10]) with mapi id 15.20.2559.012; Wed, 18 Dec 2019 14:54:42 +0000
From: Christer Holmberg <christer.holmberg@ericsson.com>
To: Lorenzo Miniero <lorenzo@meetecho.com>
CC: "mmusic@ietf.org" <mmusic@ietf.org>
Thread-Topic: [MMUSIC] Open source SIP/WebRTC real-time text integration
Thread-Index: AQHVtZMMZSSpRqWEGUi1cJqJjySxP6fADLeA///sdICAACNbAA==
Date: Wed, 18 Dec 2019 14:54:42 +0000
Message-ID: <8220BA21-0CF6-425B-80BD-7AF4F6BA6B9F@ericsson.com>
References: <20191218120451.7dd7093b@lminiero> <14EEE460-EA5C-436F-B987-003106DB01FA@ericsson.com> <20191218154807.313f9dc7@lminiero>
In-Reply-To: <20191218154807.313f9dc7@lminiero>
Accept-Language: en-US
Content-Language: en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.1e.0.191013
authentication-results: spf=none (sender IP is ) smtp.mailfrom=christer.holmberg@ericsson.com;
x-originating-ip: [89.166.49.243]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 08e435e2-f58c-40bc-98ac-08d783ca36cc
x-ms-traffictypediagnostic: DB6PR0701MB2600:
x-microsoft-antispam-prvs: <DB6PR0701MB2600AC15647CC69DE75FE73493530@DB6PR0701MB2600.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-forefront-prvs: 0255DF69B9
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(376002)(396003)(366004)(136003)(39860400002)(52314003)(53754006)(189003)(199004)(966005)(86362001)(71200400001)(478600001)(2906002)(6506007)(36756003)(66574012)(6512007)(316002)(8676002)(6486002)(76116006)(66446008)(66476007)(91956017)(66556008)(26005)(8936002)(81156014)(5660300002)(186003)(2616005)(6916009)(44832011)(33656002)(81166006)(4326008)(64756008)(66946007); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0701MB2600; H:DB6PR0701MB2421.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: nrfgopVma9Ht1dPNkUY+ZyoypX0dR+dBJ0/4QvnFpxvBE/MO2YPAnN9qMGcskNK2a3Jl/VRETMTR5N4JcJ2ViNhZMWtVg6Rhr1nbqzztF4mGPtkLR/Te+fSZYUt1xIrpwvG62dSt+bFsLbRezcypXmnb1GBuVdlYQGSxZCWBuCWWLHo/gECe59MsTimdnpuHRWZ++E/wh3IdzLdu5JM5I4E8Q4SOye2zSTrDej5gOvuP8I3lHHXThEZDvtcGsG9Hj1iLdEqOMigkqWMRDHP8vwZMnSkjaWC/FeiSQuIt91/KxuRE/NkcIX4yMB/8z5xrsUpFvWMakL57TRzfo9+zlW0D1+9WA1ShVX4emjUVqwO0GqAG1em3VSN1YqdvqF1wkizEqRGOM35wkVA9GuI1nnJjVKbTi0NriZbWFuizk7AyuHAeSfonMlrnvaXcVzaDcfi2FhVkF3bzioC1c/O7Nd7L9y8BdttU2/aZHc4QZFlQayoxeRnDg4/VkFQOOFAOxZSvwB0KthxuJZg2cCbqdLBLAht6tGVWNPpD1Q3iYvRwqktygSAKYuyYlM4MMmSE
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <1724791E5B5F5F4FAEA336ADF923133F@eurprd07.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 08e435e2-f58c-40bc-98ac-08d783ca36cc
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2019 14:54:42.0665 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tm9D5DomD0b1CuTVVIzQrumAIvk5Ugcy+LgUrCo9i91cTphYd/NaHCrGj36ptO+OrnyRsV94D2zVHQf4MWUfsolrqOZqtuxCSUy+eYxMgRw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0701MB2600
Archived-At: <https://mailarchive.ietf.org/arch/msg/mmusic/EB5ObRjL3q1i_PF9KIxNW9WZTB0>
Subject: Re: [MMUSIC] Open source SIP/WebRTC real-time text integration
X-BeenThere: mmusic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Multiparty Multimedia Session Control Working Group <mmusic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mmusic>, <mailto:mmusic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mmusic/>
List-Post: <mailto:mmusic@ietf.org>
List-Help: <mailto:mmusic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mmusic>, <mailto:mmusic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 18 Dec 2019 14:54:48 -0000

Hi,

    >> As far as browsers rejecting non-supported attributes is concerned, I
    >> think that is an issue that should be raised in W3C. As a temporary
    >> solution, could the JS app simply not remove them before passing the
    >> SDP to the browser?
    >
    > Actually I should clarify that it's what I *thought* would happen, as
    > I've seen it happening in the past in some cases. Anyway, I just tried
    > injecting some of those attributes in the SDP Janus sends to the
    > browser, i.e.:
    >
    >	a=dcmap:1 label="JanusDataChannel";subprotocol="t140"
    >	a=dcsa:1 hlang-send:en
    >	a=dcsa:1 hlang-recv:en
    >
    > and to my pleasant surprise Chrome didn't complain, so I was wrong in
    > that regard.

That's good news!

 > It doesn't do anything with those attributes either,
 > though, which probably means they're completely ignored at the moment.

Sure, that is expected.

>    There's no trace of those in the answer, for instance, which means at
>    the moment the web application would have to munge the SDP to add those
>    values manually, something I try to avoid whenever possible.
>    
>    That said, since the SIP endpoint I'm using for testing doesn't provide
>    any hint on what should be added there (e.g., in terms of supported
>    languages), my server wouldn't be able to come up with something to
>    negotiate there anyway. For labels, since this first integration only
>    supports 1-1 calls, I thought it safe to rely on the default label that
>    is opened when data channels are negotiated with Janus (the
>    "JanusDataChannel" you see listed in the snippet above).
  
That's fine - the T.140 data channel usage does not specify a label value, only the sub-protocol value ("t140").

Regards,

Christer



    > On 18/12/2019, 13.05, "mmusic on behalf of Lorenzo Miniero"
    > <mmusic-bounces@ietf.org on behalf of lorenzo@meetecho.com> wrote:
    > 
    >     Hi all,
    >     
    >     as I anticipated in another post, I've recently started working
    > with real-time text: mostly for fun (I've always been intrigued by the
    >     protocol), but also because I wanted to get it working with
    > WebRTC, especially in light of its importance for the upcoming NG
    > emergency services. Considering Christer's recent efforts on
    > standardizing how to use RTT on WebRTC, I thought I'd share this
    > here, so that it can hopefully provide a testing framework: it is not
    > complete, and doesn't adhere 100% to the specification yet (more on
    > that in a minute), but I think it might be helpful as a running code
    > reference anyway. 
    >     Specifically, I've integrated support for real-time text in our
    > open source Janus WebRTC server. Janus is a modular component, and
    > one of the plugins implements a SIP gateway: this is where I
    > implemented support for T.140 and red. The way it works is
    > "straightforward": 
    >     	1. Any time we get an incoming SIP call on the SIP side
    > with an m=text m-line, we translate that to an m=application, so that
    >     	we can negotiate data channels on the WebRTC side.
    >     
    >     	2. At the same time, if the WebRTC peer negotiates an
    >     	m=application line, we turn that into an m=text section to
    >     	negotiate real-time text on the SIP side.
    >     
    >     	3. Then, if we receive T.140 packets from SIP, we relay
    > them via data channels using binary data (we use an Uint8Array in
    >     	the web application); if we receive RED packets, we parse
    > it in order to get the T.140 blocks we need, and relay those via data
    >     	channels (since the draft explains redundancy is not
    > needed, thanks to SCTP).
    >     
    >     	4. When we get T.140 blocks via data channels (which
    > again the web application sends as an Uint8Array), we either send
    > them as they are in RTP packets (if RED was not negotiated), or we
    >     	create a RED RTP packet with redundant info on previously
    > sent packets (if RED was negotiated instead).
    >     
    >     This is the effort in a nutshell, and from my simple tests it
    > seems to be working as expected so far. You can find a more
    > comprehensive description of the whole effort in this blog post:
    >     
    >     https://protect2.fireeye.com/v1/url?k=c2a19457-9e284e10-c2a1d4cc-0cc47ad93c18-c1355b6e18f62d8a&q=1&e=78bf43a6-cc22-44d8-bbaf-2cb644177cfa&u=https%3A%2F%2Fwww.meetecho.com%2Fblog%2Frealtime-text-sip-and-webrtc%2F
    >     
    >     The Janus branch supporting real-time text, instead, is here:
    >     
    >     https://protect2.fireeye.com/v1/url?k=ce273934-92aee373-ce2779af-0cc47ad93c18-c8ae4c94a2ec0095&q=1&e=78bf43a6-cc22-44d8-bbaf-2cb644177cfa&u=https%3A%2F%2Fgithub.com%2Fmeetecho%2Fjanus-gateway%2Fpull%2F1898
    >     
    >     As explained in the blog post, the effort is not complete, and
    > there are some things that are either missing, or need to be
    > improved, namely: 
    >     	1. We're not using the dcmap and dcsa attributes on the
    > WebRTC side yet: the reason is that browsers don't support them, at
    >     	the moment, and so putting them there may result in the
    > SDP being rejected and the session broken. This means that,
    >     	currently, we use a default label for exchanging T.140
    > blocks with the server.
    >     
    >     	2. While we support RED, we're not using the redundant
    > info for packets we receive yet, and we're not properly handling
    > packet loss or out of order packets yet either. This is something we
    >     	plan to work on in the future, as I was more interested in
    >     	getting the specification in general to work first.
    >     
    >     	3. On the client side in WebRTC, we're not doing any
    > buffering, meaning we send every character as soon as we type it,
    > which is clearly suboptimal. Again, something we plan to fix later on,
    >     	either in the web application (buffer there), or on the
    > server side (buffer incoming T.140 blocks there, before crafting RTP
    >     	packets).
    >     
    >     I hope this will be considered useful, and I'm looking forward to
    > keep on working on this as the specifications moves forward. If you
    > have any questions or doubts, please don't hesitate to ask; besides,
    > I'll be in Vancouver for the next IETF, so in case you want to talk
    > about it in person there, see a demo, or make some interoperability
    > tests, I'll be glad to do that as well.
    >     
    >     Thanks,
    >     Lorenzo
    >     
    >     -- 
    >     I'm getting older but, unlike whisky, I'm not getting any better
    >     https://twitter.com/elminiero
    >     
    >     _______________________________________________
    >     mmusic mailing list
    >     mmusic@ietf.org
    >     https://www.ietf.org/mailman/listinfo/mmusic
    >     
    > 
    
    
    
    -- 
    I'm getting older but, unlike whisky, I'm not getting any better
    https://twitter.com/elminiero