Re: [Hls-interest] Use of characteristic tags for Subtitles in HLS streaming

["May, Bill" <[e-mail]>]

Eryk,

Are the characteristics required for automatically selecting when accessibility settings request SDH?

We have some subtitles that are like English (without any characteristics), and some English[CC] (with transcribes-spoken-dialog and describes-music-and-sound).

If there are no characteristics, are the subtitles automatically selected with the accessibility settings are set?

Thanks,
Bill

On Jul 23, 2021, at 8:31 AM, Eryk Vershen <evershen=[e-mail]<mailto:evershen=[e-mail]>> wrote:

Thanks for your inquiry.

I can see how the Authoring Specification rule 4.5 might cause confusion. The intent of the rule is that "public.accessibility.describes-music-and-sound" should be used in addition to "public.accessibility.transcribes-spoken-dialog". I'll make sure the rule is revised to be clearer.

The three characteristics relevant to subtitle renditions are (as draft-pantos-hls-rfc8216bis-09 says):
"public.accessibility.transcribes-spoken-dialog"
"public.accessibility.describes-music-and-sound"
"public.easy-to-read"

Characteristics are also known as AVMediaCharacteristics (see https://developer.apple.com/documentation/avfoundation/avmediacharacteristic<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fdocumentation%2Favfoundation%2Favmediacharacteristic&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138327352%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sKB6vwlkvOKi0wwAY3wyAMv2bv4iUgpcaLgvsZgSdGk%3D&reserved=0>).

A subtitle rendition may have none of these. For example, if only portions of the dialog are subtitled.
If the rendition is subtitled throughout, it should be marked "public.accessibility.transcribes-spoken-dialog".
If, in addition, the rendition has sound effects & music described, then it should also be marked "public.accessibility.describes-music-and-sound"

The last characteristic ("public.easy-to-read") is defined in the AVFoundation documentation above (AVMediaCharacteristicEasyToRead) as "'easy reader' captions (per the CEA-608 specification)".

regards,
-eryk

On Jul 22, 2021, at 8:44 AM, James Hart <[e-mail]<mailto:[e-mail]>> wrote:

Hello,

I am trying to determine the correct use of the characteristics tags for subtitles in HTTP LIve Streaming and i am a little confused by the various sources of information and hoping some clarifications can be provided on what devices require:

>From RFC-8216-09<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-pantos-hls-rfc8216bis-09%23section-4.4.6.1&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138337307%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WhHvxmLUoNnVDUq3H8k2YMtQloaBSpOKQ%2BzJ7YllLYU%3D&reserved=0> i see:

A SUBTITLES Rendition MAY include the following characteristics:
"public.accessibility.transcribes-spoken-dialog", public.accessibility.describes-music-and-sound", and
"public.easy-to-read" (which indicates that the subtitles have been edited for ease of reading).


>From the HLS Authoring Specification<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fdocumentation%2Fhttp_live_streaming%2Fhls_authoring_specification_for_apple_devices&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138337307%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rlXRj3C9WPJ0FVi62yALi8gWk%2BBfYP%2Fuk1L07olHoOs%3D&reserved=0> for Apple Devices i see only one characteristic defined:


4.5. If a subtitles track is intended to provide accessibility for people who are deaf or hard of hearing, it MUST be marked with the attribute CHARACTERISTICS="public.accessibility.describes-music-and-sound". (Subtitles with this attribute value are treated the same as closed captions.)

Then if i compare this with the sample streams on the Developer Site<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fstreaming%2Fexamples%2F&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MPSaRKDfy1NMENPtttujtqOSVzQuMR2s3aYAQo5Le0w%3D&reserved=0> i can see one such example contains subtitles <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fstreaming%2Fexamples%2Fbasic-stream-osx-ios5.html&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZqYNWBUK7gRD6dgTTljbZrFd5Z69j7xGbrmJrqguA8w%3D&reserved=0> with the characteristics "public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound" within the top level m3u8, which does not appear aligned fully with the HLS authoring specification.  For reference this was the example i used:

$ curl 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevstreaming-cdn.apple.com%2Fvideos%2Fstreaming%2Fexamples%2Fbipbop_16x9%2Fbipbop_16x9_variant.m3u8&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UWi64zEVPZZcG7Mmj4P31D99qUTL0UX7EmYBlsK03yE%3D&reserved=0>'
#EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac/prog_index.m3u8"


#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
< - - snip - - >


Appreciate some pointers on the correct signaling here.

Many thanks,

James Hart
--
Hls-interest mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/hls-interest

--
Hls-interest mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/hls-interest