IETF Logo Mail Archive

Re: [Hls-interest] coming to Apple HLS clients later this year: M-JPEG support

Roger Pantos <rpantos@apple.com> Wed, 02 March 2022 21:31 UTC

Return-Path: <rpantos@apple.com>
X-Original-To: hls-interest@ietfa.amsl.com
Delivered-To: hls-interest@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 535943A088F for <hls-interest@ietfa.amsl.com>; Wed, 2 Mar 2022 13:31:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.108
X-Spam-Level:
X-Spam-Status: No, score=-2.108 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7j50tpSWBFkZ for <hls-interest@ietfa.amsl.com>; Wed, 2 Mar 2022 13:31:06 -0800 (PST)
Received: from ma1-aaemail-dr-lapp02.apple.com (ma1-aaemail-dr-lapp02.apple.com [17.171.2.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 320AB3A0892 for <hls-interest@ietf.org>; Wed, 2 Mar 2022 13:31:06 -0800 (PST)
Received: from pps.filterd (ma1-aaemail-dr-lapp02.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp02.apple.com (8.16.0.42/8.16.0.42) with SMTP id 222LF1dt049368; Wed, 2 Mar 2022 13:31:05 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=3yUnzzgcr5MDDAPIWL0nENLOJ9xBncAXlb6KPEccVMg=; b=AqgVCjgi0dJSU6hU8uE8G/UTSWQSnz5CQ2WJjbYA1L4jDXFtcWLL+lBza2FCJR2lySJ+ aR2VqsjTmwdtM2LKeiHXLFWwUxINFT2po5nOVLirT+nZF+JHtMOoGB5AeLnDxYxSVIMv oad9yuYo/Z2n0ii7IjPn/WxaJsxvaGsXJvADrsDeCLvx9r7DLFvo83pSbSpaSNemeuVu zG/J0I7Zt1MiB4zZVFqMgMaMZsPcX13r2fU1igsO/aljwgWNKJXAY5FJsGf3JlDqs+BH eQr18Y/FjUpVPjb+qVlslvhd3YbVpQM5K2cUaz9XXkUT0mI0YwWzegNLDfTvoiLFncGE zw==
Received: from rn-mailsvcp-mta-lapp02.rno.apple.com (rn-mailsvcp-mta-lapp02.rno.apple.com [10.225.203.150]) by ma1-aaemail-dr-lapp02.apple.com with ESMTP id 3efhrxd1qk-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 02 Mar 2022 13:31:05 -0800
Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPS id <0R8400D9PZRSEC20@rn-mailsvcp-mta-lapp02.rno.apple.com>; Wed, 02 Mar 2022 13:31:04 -0800 (PST)
Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) id <0R8400300ZRLDI00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Wed, 02 Mar 2022 13:31:04 -0800 (PST)
X-Va-A:
X-Va-T-CD: 81ca60fce39c2560b6c4a7e5841f9b8f
X-Va-E-CD: 18fa313a53dff409db6c2488b7e792d9
X-Va-R-CD: 929e8339eeb80c3a2cedcd4e479e2aa7
X-Va-CD: 0
X-Va-ID: db3dc7ce-fa11-4b48-8344-2374e7984baa
X-V-A:
X-V-T-CD: 81ca60fce39c2560b6c4a7e5841f9b8f
X-V-E-CD: 18fa313a53dff409db6c2488b7e792d9
X-V-R-CD: 929e8339eeb80c3a2cedcd4e479e2aa7
X-V-CD: 0
X-V-ID: 5ff86fe4-4389-4740-bcfb-d5033345e510
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.816 definitions=2022-03-02_05:2022-02-26, 2022-03-02 signatures=0
Received: from smtpclient.apple ([17.11.179.42]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPSA id <0R8400S4NZRREU00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Wed, 02 Mar 2022 13:31:04 -0800 (PST)
From: Roger Pantos <rpantos@apple.com>
Message-id: <C3F8180C-0AD3-4365-B3B4-88BC56CB6590@apple.com>
Content-type: multipart/alternative; boundary="Apple-Mail=_D966CA59-A10D-409C-AB9C-6E04D28F9594"
MIME-version: 1.0 (Mac OS X Mail 16.0 \(3696.80.82.1.1\))
Date: Wed, 02 Mar 2022 13:31:03 -0800
In-reply-to: <DM6PR14MB2825422815CDB6E8192E35B2EF039@DM6PR14MB2825.namprd14.prod.outlook.com>
Cc: "hls-announce@lists.apple.com" <hls-announce@lists.apple.com>, "hls-interest@ietf.org" <hls-interest@ietf.org>
To: "Galluccio, Robert (NBCUniversal)" <Robert.Galluccio@nbcuni.com>
References: <5EDBFCDA-9040-4443-9694-C5D7E7F79C1C@apple.com> <CAF-MBSJq3MQ-oeOeed9BgMmn8BR+9QX2-YSpbq1vn=w3eTK_Jw@mail.gmail.com> <A7C0E64D-7260-4F0D-96F2-27963D2ACF5A@apple.com> <CAF-MBS+5VPHvn_Yw1t4JrSgp2N_3ZmHG-h2M9WTRHNgOSk1L=Q@mail.gmail.com> <DM6PR14MB2825422815CDB6E8192E35B2EF039@DM6PR14MB2825.namprd14.prod.outlook.com>
X-Mailer: Apple Mail (2.3696.80.82.1.1)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.816 definitions=2022-03-02_05:2022-02-26, 2022-03-02 signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/hls-interest/Vdicr0maW6DGTPNdhTCWWyORxsY>
Subject: Re: [Hls-interest] coming to Apple HLS clients later this year: M-JPEG support
X-BeenThere: hls-interest@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions about HTTP Live Streaming \(HLS\)." <hls-interest.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hls-interest>, <mailto:hls-interest-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/hls-interest/>
List-Post: <mailto:hls-interest@ietf.org>
List-Help: <mailto:hls-interest-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hls-interest>, <mailto:hls-interest-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 02 Mar 2022 21:31:11 -0000


> On Mar 2, 2022, at 1:20 PM, Galluccio, Robert (NBCUniversal) <Robert.Galluccio@nbcuni.com> wrote:
> 
> Hi Roger,
>  
> Is there any consideration for introducing grids of multiple thumbnails expressed in one M-JPEG segment and therefore taking up a larger EXTINF value?
>  
> This would be similar to DASH-IF-IOP 11.5:
> https://dashif-documents.azurewebsites.net/DASH-IF-IOP/master/DASH-IF-IOP.html#codecs-thumbnails <https://dashif-documents.azurewebsites.net/DASH-IF-IOP/master/DASH-IF-IOP.html#codecs-thumbnails>
>  
> And also to the custom tags defined on this Roku developer documentation page:
> https://developer.roku.com/en-gb/docs/developer-program/media-playback/trick-mode/hls-and-dash.md#ext-x-tiles <https://developer.roku.com/en-gb/docs/developer-program/media-playback/trick-mode/hls-and-dash.md#ext-x-tiles>
>  
> This can provide network round trip savings for VOD assets where the entire timeline is already defined when loading the asset.

For VOD content it is pretty common to compose multiple fMP4 segments into a single MPEG-4 resource using byte range addressing. That allows clients to download all thumbnails using a single HTTP request (or to download them progressively as needed, if the client prefers).

>  
> And separately, with regards to Apple HLS clients support, will there be a plan to expose obtaining the thumbnail images outside of AVKit (e.g. asking the system for a thumbnail image for a given playback time)?

Yes, this is already available (for unencrypted thumbnails) using the AVAssetImageGenerator API.


Roger.

>  
> Best,
> Rob

v2.23.0 | Report a Bug | By Email