Re: [Hls-interest] Use of characteristic tags for Subtitles in HLS streaming

[Eryk Vershen <[e-mail]>]

I believe the effect of the setting is to "prefer closed captions and SDH". That is, ordinary subtitles will be used if that is all that is available.
However, this will cause closed captions to be preferred over ordinary subtitles.

One of my co-workers pointed out that I should have referenced AVMediaFormat.h in addition to the documentation.

The discussion in AVMediaFormat.h for AVMediaCharacteristicTranscribesSpokenDialogForAccessibility (i.e., "public.accessibility.transcribes-spoken-dialog") states that you should not use the characteristic unless it "identifies speakers whenever other visual cues are insufficient for a viewer to determine who is speaking".

-eryk


> On Jul 23, 2021, at 8:38 AM, May, Bill <Bill.May=[e-mail]> wrote:
> 
> Eryk, 
> 
> Are the characteristics required for automatically selecting when accessibility settings request SDH?
> 
> We have some subtitles that are like English (without any characteristics), and some English[CC] (with transcribes-spoken-dialog and describes-music-and-sound).
> 
> If there are no characteristics, are the subtitles automatically selected with the accessibility settings are set?
> 
> Thanks, 
> Bill
> 
>> On Jul 23, 2021, at 8:31 AM, Eryk Vershen <evershen=[e-mail] <mailto:evershen=[e-mail]>> wrote:
>> 
>> Thanks for your inquiry.
>> 
>> I can see how the Authoring Specification rule 4.5 might cause confusion. The intent of the rule is that "public.accessibility.describes-music-and-sound" should be used in addition to "public.accessibility.transcribes-spoken-dialog". I'll make sure the rule is revised to be clearer.
>> 
>> The three characteristics relevant to subtitle renditions are (as draft-pantos-hls-rfc8216bis-09 says):
>> "public.accessibility.transcribes-spoken-dialog"
>> "public.accessibility.describes-music-and-sound"
>> "public.easy-to-read"
>> 
>> Characteristics are also known as AVMediaCharacteristics (see https://developer.apple.com/documentation/avfoundation/avmediacharacteristic <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fdocumentation%2Favfoundation%2Favmediacharacteristic&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138327352%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sKB6vwlkvOKi0wwAY3wyAMv2bv4iUgpcaLgvsZgSdGk%3D&reserved=0>).
>> 
>> A subtitle rendition may have none of these. For example, if only portions of the dialog are subtitled.
>> If the rendition is subtitled throughout, it should be marked "public.accessibility.transcribes-spoken-dialog".
>> If, in addition, the rendition has sound effects & music described, then it should also be marked "public.accessibility.describes-music-and-sound"
>> 
>> The last characteristic ("public.easy-to-read") is defined in the AVFoundation documentation above (AVMediaCharacteristicEasyToRead) as "'easy reader' captions (per the CEA-608 specification)".
>> 
>> regards,
>> -eryk
>> 
>>> On Jul 22, 2021, at 8:44 AM, James Hart <[e-mail] <mailto:[e-mail]>> wrote:
>>> 
>>> Hello,
>>> 
>>> I am trying to determine the correct use of the characteristics tags for subtitles in HTTP LIve Streaming and i am a little confused by the various sources of information and hoping some clarifications can be provided on what devices require:
>>> 
>>> From RFC-8216-09 <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-pantos-hls-rfc8216bis-09%23section-4.4.6.1&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138337307%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WhHvxmLUoNnVDUq3H8k2YMtQloaBSpOKQ%2BzJ7YllLYU%3D&reserved=0> i see:
>>> 
>>> A SUBTITLES Rendition MAY include the following characteristics:
>>> "public.accessibility.transcribes-spoken-dialog", public.accessibility.describes-music-and-sound", and
>>> "public.easy-to-read" (which indicates that the subtitles have been edited for ease of reading).
>>> 
>>> From the HLS Authoring Specification <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fdocumentation%2Fhttp_live_streaming%2Fhls_authoring_specification_for_apple_devices&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138337307%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rlXRj3C9WPJ0FVi62yALi8gWk%2BBfYP%2Fuk1L07olHoOs%3D&reserved=0> for Apple Devices i see only one characteristic defined:
>>> 
>>> 4.5. If a subtitles track is intended to provide accessibility for people who are deaf or hard of hearing, it MUST be marked with the attribute CHARACTERISTICS="public.accessibility.describes-music-and-sound". (Subtitles with this attribute value are treated the same as closed captions.)
>>> 
>>> Then if i compare this with the sample streams on the Developer Site <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fstreaming%2Fexamples%2F&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MPSaRKDfy1NMENPtttujtqOSVzQuMR2s3aYAQo5Le0w%3D&reserved=0> i can see one such example contains subtitles  <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Fstreaming%2Fexamples%2Fbasic-stream-osx-ios5.html&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZqYNWBUK7gRD6dgTTljbZrFd5Z69j7xGbrmJrqguA8w%3D&reserved=0>with the characteristics "public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound" within the top level m3u8, which does not appear aligned fully with the HLS authoring specification.  For reference this was the example i used:
>>> 
>>> $ curl 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8 <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevstreaming-cdn.apple.com%2Fvideos%2Fstreaming%2Fexamples%2Fbipbop_16x9%2Fbipbop_16x9_variant.m3u8&data=04%7C01%7Cbill.may%40disneystreaming.com%7C556ae618d588410189b508d94deefe56%7C65f03ca86d0a493e9e4ac85ac9526a03%7C1%7C0%7C637626511138347260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UWi64zEVPZZcG7Mmj4P31D99qUTL0UX7EmYBlsK03yE%3D&reserved=0>'
>>> #EXTM3U
>>>  
>>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
>>> #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac/prog_index.m3u8"
>>>  
>>>  
>>> #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
>>> < - - snip - - >
>>> 
>>> 
>>> Appreciate some pointers on the correct signaling here.
>>> 
>>> Many thanks,
>>> 
>>> James Hart 
>>> -- 
>>> Hls-interest mailing list
>>> [e-mail] <mailto:[e-mail]>
>>> https://www.ietf.org/mailman/listinfo/hls-interest <https://www.ietf.org/mailman/listinfo/hls-interest>
>> 
>> -- 
>> Hls-interest mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/hls-interest
> 
> -- 
> Hls-interest mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hls-interest