[Hls-interest] Image-based subtitles and trickplay tracks

"Weil, Nicolas" <nicoweil@elemental.com> Wed, 20 May 2020 22:13 UTC

Return-Path: <prvs=4025c6887=nicoweil@elemental.com>
X-Original-To: hls-interest@ietfa.amsl.com
Delivered-To: hls-interest@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F2A53A07C3 for <hls-interest@ietfa.amsl.com>; Wed, 20 May 2020 15:13:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gSIJOQagWa_J for <hls-interest@ietfa.amsl.com>; Wed, 20 May 2020 15:13:06 -0700 (PDT)
Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C03CB3A07C6 for <hls-interest@ietf.org>; Wed, 20 May 2020 15:13:06 -0700 (PDT)
IronPort-SDR: ygkfO1ae/6x9HhzXOQzLgAWTn6Zx51XEx2VOEkPCr1W0T5pMdh+6N4ydg5nnX32AtHXHdbGzWQ zQ85nw/Ej9vA==
X-IronPort-AV: E=Sophos; i="5.73,415,1583193600"; d="scan'208,217"; a="44823246"
Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 20 May 2020 22:13:04 +0000
Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com (Postfix) with ESMTPS id 5BBB3A1CAF for <hls-interest@ietf.org>; Wed, 20 May 2020 22:13:03 +0000 (UTC)
Received: from EX13D02EUB001.ant.amazon.com (10.43.166.150) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 May 2020 22:13:03 +0000
Received: from EX13D02EUB001.ant.amazon.com (10.43.166.150) by EX13D02EUB001.ant.amazon.com (10.43.166.150) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 May 2020 22:13:02 +0000
Received: from EX13D02EUB001.ant.amazon.com ([10.43.166.150]) by EX13D02EUB001.ant.amazon.com ([10.43.166.150]) with mapi id 15.00.1497.006; Wed, 20 May 2020 22:13:02 +0000
From: "Weil, Nicolas" <nicoweil@elemental.com>
To: "hls-interest@ietf.org" <hls-interest@ietf.org>
Thread-Topic: Image-based subtitles and trickplay tracks
Thread-Index: AdYu8zEvfSZypB1NTpWZomh1td5GPw==
Date: Wed, 20 May 2020 22:12:44 +0000
Deferred-Delivery: Wed, 20 May 2020 22:11:51 +0000
Message-ID: <4959700c860e48079af52488074e2236@EX13D02EUB001.ant.amazon.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.43.166.33]
Content-Type: multipart/alternative; boundary="_000_4959700c860e48079af52488074e2236EX13D02EUB001antamazonc_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/hls-interest/3Igre5IelQQSkD2oSX-UfjEb9as>
Subject: [Hls-interest] Image-based subtitles and trickplay tracks
X-BeenThere: hls-interest@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions about HTTP Live Streaming \(HLS\)." <hls-interest.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hls-interest>, <mailto:hls-interest-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/hls-interest/>
List-Post: <mailto:hls-interest@ietf.org>
List-Help: <mailto:hls-interest-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hls-interest>, <mailto:hls-interest-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 20 May 2020 22:13:09 -0000

Hello,

We are often seeing two image-related topics causing interoperability problems as they are not currently covered by the HLS spec. Normalizing the implementations around an official specification for these two points would be great:

Image-based subtitles tracks
For workflow reasons and charset reasons, some content owners don't include text-based subtitles in the live channels sources that they provide to distributors, but rather image-based subtitles (like DVB-Sub). While it's possible to transform these subtitles as IMSC1 Image Profile as per DASH-IF IOP section 6.4.4, there is no equivalent IMSC1 Image Profile support in the HLS RFC, which means that companies will continue to rely on proprietary forks of the HLS RFC to support these use cases. Even if it wasn't supported by Apple players, it would be tremendously helpful for interoperability in the rest of the HLS ecosystem.

Image-based trickplay tracks
For player resources optimization reasons, the use of a video track as a trickplay artefact is not always possible, and a lot of player providers recommend the use of image thumbnails tracks instead of special low framerate video tracks. DASHIF IOP section 6.2.6 covers this use case but there is equivalent support in the HLS RFC. There is the Image Media playlists HLS extension proposal from Roku/Disney/WarnerMedia here https://github.com/image-media-playlist/spec but its relevance/adoption is currently limited by the fact that it's not part of the RFC. Same logic here: even if not supported by Apple players which don't need it as they can leverage I-frame tracks, it would be super useful for the rest of the HLS ecosystem to get this officially part of the RFC.

Thanks for your consideration,
Nicolas

----------------
Nicolas Weil | Senior Product Manager - Media Services
AWS Elemental