Re: [rtcweb] WebRTC and Real-time Translation

One of the key questions for "Next Version Use Cases" is what
WebRTC-deficiencies are preventing these use cases from being
satisfactorily implemented today.

For example, speech transcription cloud services have been implemented over
Websockets, where a snippet of speech is uploaded, and a transcription is
provided in reply.  The latency is satisfactory for some uses cases.
Improvements can perhaps be made by sending an audio stream and receiving a
transcription via the data channel, but this is also within the
capabilities of the existing RTCWEB protocols and WebRTC-PC API.

What seems to differentiate *next version* scenarios are situations where
the processing is best done on the device, in order to lower latency or
enhance privacy.  On-device processing brings in discussion of
workers/worklets, access to raw audio/video, etc.  However, so far I'm not
aware of on-device implementations of transcription or translation.

On Wed, Sep 26, 2018 at 6:10 PM Adam Sobieski <adamsobieski@hotmail.com>
wrote:

> IETF RTCWEB Working Group,
>
>
>
> Greetings. I opened an issue on *WebRTC and Real-time Translation* at the
> GitHub repository for WebRTC version next use cases (
> https://github.com/w3c/webrtc-nv-use-cases/issues/2).
>
>
> Introduction
>
> Real-time translation is both an interesting and important use case for a
> next version of WebRTC.
> Speech Recognition, Translation and Speech Synthesis
>
> Approaches to real-time speech-to-speech machine translation include those
> which interconnect speech recognition, translation and speech synthesis
> components and services. In that regard, we can consider client-side,
> on-prem, server-side, third-party and cloud-based components and services.
> In that regard, we can also consider both free and priced components and
> services.
>
> We can envision *post-text* speech technology and machine translation
> components and services. Speech recognition need not output to text; we can
> consider speech-to-SSML. Machine translation need not input from nor output
> to text; we can consider SSML-to-SSML machine translation. Components and
> services may provide various options with respect to their input and output
> data formats.
> Connecting Components and Services by Constructing Graphs
>
> We can consider APIs which facilitate the construction of graphs which
> represent the flow of data between components and services. As these graphs
> are constructed, users could be apprised of relevant notifications,
> requests for permissions and options for payments. As these constructed
> graphs are activated, a number of protocols could be utilized to
> interconnect the components and services which, together, provide users
> with real-time translation.
> Hyperlinks
>
> WebRTC Translator Demo <https://www.youtube.com/watch?v=Tv8ilBOKS2o>
> Real Time Translation in WebRTC
> <https://www.youtube.com/watch?v=EPBWR_GNY9U>
>
>
>
>
>
> Best regards,
>
> Adam Sobieski
>
> http://www.phoster.com/contents/
>
>
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb
>