Re: [MMUSIC] New Version Notification for draft-abhishek-mmusic-superimposition-grouping-00.txt(Internet mail)

[Paul Kyzivat <[e-mail]>]

On 1/17/21 5:29 PM, rabhishek(RohitAbhishek) wrote:
> Hi Paul, Christer,
> 
> My understanding is the transparent attribute is same as alpha compositing which would control the linear interpolation between the foreground and the background pixels. Here’s an example wrt figure 1 in the draft.

First, I'm concerned with you starting your reply with "my understanding 
is ...". This is your draft! The goal is to ensure that all readers come 
to the same understanding. There should be no vagueness.

Another thing: Section 5 says:

        transparency-attribute =
                     "a=transparency:" transparency-tag
        transparency-tag =tranparency-value *("," tranparency-value) CRLF
        transparency-value= alpha

    Alpha describes the transparency for the foreground media stream.  It
    is identified by its transparency-tag values in the transparency-
    attribute.  It could be an integer with values between 0 and 100.
    This is an informative value.  Details of interpretion to be left
    open to the renderer, expect that a value of 0 means foreground media
    is opaque and value of 100 means that it is transparent.

 From an ABNF perspective "alpha" is undefined. The text says it *could 
be" an integer between 0 and 100, but "details of interpretation are 
open to the renderer". That negates the value of creating a standard. 
The whole point of a standard is to ensure that interoperating parties 
handle things the same way.

> X= Background Image
> A, B, C= foreground image (Media C partially over B)
> Transparency(to be applied each pixel) =alpha(A), beta (B), gamma (C)
> Background media which shows through A= (1-alpha)X
> Background media which shows through B and C = (1-beta)( 1-gamma)X
> Overall composed image= gamma.C+ (1-gamma)[beta.B+ (1- beta.B)X]

I don't know much about this technology, so perhaps this would be well 
known to implementers. But it shouldn't just be assumed. If this is then 
intended mechanism then it should be specified in the document, either 
directly or by reference to some public document.

Finally, ISTM that you don't need to treat the background document as a 
special case. The grouping mechanism can be used to identify all the 
media streams participating in this superposition mechanism, including 
the one you are considering to be the background.

IIUC the scheme you describe defines the result of superposition based 
on the transparency values of the individual streams, so that no other 
ordering mechanism is needed. (Do I have that right?) If so, isn't the 
background just the media stream with tranparency = 100?

	Thanks,
	Paul

> Best Regards,
> Rohit
> 
> On 1/14/21, 9:25 AM, "mmusic on behalf of Paul Kyzivat" <[e-mail] on behalf of [e-mail]> wrote:
> 
>      On 1/14/21 4:54 AM, Christer Holmberg wrote:
>      > Hi,
>      >
>      >> One more thing, regarding transparency:
>      >>
>      >> I'm fuzzy on how this is intended to work. I would think that each superposition layer would have some parts that are fully transparent.
>      >> (Transparent pixels.) And then the transparency attribute would apply to the parts that aren't fully transparent. But that is only a guess. I think you need to say more about how the superposition is supposed to work.
>      >
>      > Isn't "transparent background" information carried within the video stream (codec or container) itself?
> 
>      That is my impression, but I don't really know. But the transparency
>      attribute must do *something*.
> 
>      The draft needs to explain better.
> 
>      	Thanks,
>      	Paul
> 
>      > Regards,
>      >
>      > Christer
>      >
>      >
>      >
>      > On 12/14/20 11:21 AM, Paul Kyzivat wrote:
>      >> Rohit,
>      >>
>      >> Here are a few more points I forgot:
>      >>
>      >> 1) syntax
>      >>
>      >> In section 3 you repeat the syntax for the 'mid' attribute from RFC5888.
>      >> I guess you don't intend it to be normative, but it gave me that
>      >> impression.
>      >>
>      >> I don't think you need to repeat the syntax here. It should be
>      >> sufficient to mention that you follow RFC5888 in use of the 'mid'
>      >> attribute to identify the media sections that are to be included in
>      >> the superposition.
>      >>
>      >> Similarly in section 5, I suggest that you only provide ABNF for the
>      >> additions to syntax required for your extension. Specifically:
>      >>
>      >>     semantics =/ "superposition" <semantics as defined in RFC5888>
>      >>
>      >> (Everything else is inherited from RFC5888.)
>      >>
>      >> 2) applicability
>      >>
>      >> Regarding the following in section 3:
>      >>
>      >>      ... An application that receives a session
>      >>      description that contains "m" lines grouped together using "S"
>      >>      semantics MUST superimpose the corresponding media streams on top
>      >> of
>      >>      the background media stream.
>      >>
>      >> You need to recognize that because this feature is an extension its
>      >> requirements can only apply to those that implement the extension. Its
>      >> best to call that out specifically.
>      >>
>      >> Also, please give some thought to what will happen a device supporting
>      >> this negotiates a session with a device that doesn't. The
>      >> non-supporting device will treat these as independent media streams.
>      >> If that is a problem for you then we need to discuss strategies for
>      >> avoiding that situation.
>      >>
>      >> 3) differing size/resolution
>      >>
>      >> How the overlaying will work seems well defined if the image width and
>      >> height in pixels and pixel shape is the same for all the media streams
>      >> in the superposition group. But what if they aren't?
>      >>
>      >> I think the document should specify what should happen in such cases.
>      >>
>      >>       Thanks,
>      >>       Paul
>      >>
>      >> On 12/13/20 3:54 PM, Paul Kyzivat wrote:
>      >>> On 12/12/20 2:06 PM, Christer Holmberg wrote:
>      >>>> Hi,
>      >>>>
>      >>>> A few comments (some of which I gave already previously):
>      >>>
>      >>> Christer covered most of what I wanted to say. I'll just add a few
>      >>> tweaks below.
>      >>>
>      >>>> Q1:
>      >>>>
>      >>>> In Section 1, where you talk about the limitations of the mechanism,
>      >>>> you should mention that CLUE must be used something more "fancy" is
>      >>>> needed.
>      >>>>
>      >>>> ---
>      >>>>
>      >>>> Q2:
>      >>>>
>      >>>> In Section 3, you require a specific order of the m- lines in the
>      >>>> SDP. I think SDP explicitly says that the order of m- lines is not
>      >>>> relevant. Also, I know that parsers may change the orders of the m-
>      >>>> lines.
>      >>>>
>      >>>> (I also think that any counting should start from 1, not 0)
>      >>>>
>      >>>> My suggestion would be to have an explicit "order" attribute instead.
>      >>>
>      >>> I think it would be acceptable to use the order of the tokens in the
>      >>> a:group line to define the ordering of the "layers". The "lowest"
>      >>> layer would then be the background on which the others are overlain.
>      >>> (Whether the lowest layer is the first or last in the list is TBD.)
>      >>>
>      >>>> ---
>      >>>>
>      >>>> Q3:
>      >>>>
>      >>>> The example in Section 6 talks about "a background video". But, this
>      >>>> background video is not described in the SDP.
>      >>>>
>      >>>> Related to that, if the SDP describes multiple "background videos",
>      >>>> you need to describe how you map superimposed videos to a specific
>      >>>> background video.
>      >>>>
>      >>>> Based on the text in Section 3 (see Q2), I am not sure whether the
>      >>>> 0th m- line is supposed to be the background video, but the text in
>      >>>> Section 6 says that both m- lines are for superimposed video streams.
>      >>>
>      >>> See my comment above.
>      >>>
>      >>> The background layer would presumably have a transparency of zero.
>      >>>
>      >>> (It would also be good to specify that if the transparency attribute
>      >>> is omitted it defaults to zero.)
>      >>>
>      >>>> ---
>      >>>>
>      >>>> Q4:
>      >>>>
>      >>>> What happens to the superimposed videos if I disable/remove/mute the
>      >>>> background video?
>      >>>>
>      >>>> ---
>      >>>>
>      >>>> Q5:
>      >>>>
>      >>>> This is editorial, but please call the group attribute something
>      >>>> else than just "S". "supim", "spim" or something :)
>      >>>
>      >>> Yes please. "S" is way to cryptic.
>      >>>
>      >>>       Thanks,
>      >>>       Paul
>      >>>
>      >>>> Regards,
>      >>>>
>      >>>> Christer
>      >>>>
>      >>>>
>      >>>>
>      >>>> -----Original Message-----
>      >>>> From: mmusic <[e-mail]> On Behalf Of Rohit Abhishek
>      >>>> Sent: lauantai 12. joulukuuta 2020 0.28
>      >>>> To: [e-mail]
>      >>>> Subject: [MMUSIC] FW: New Version Notification for
>      >>>> draft-abhishek-mmusic-superimposition-grouping-00.txt
>      >>>>
>      >>>> Dear All,
>      >>>>
>      >>>> A new version of the draft “SDP Superimposition Grouping framework”
>      >>>> has been uploaded.
>      >>>> We have changed the name from  “SDP Overlay Grouping framework for
>      >>>> immersive telepresence media streams” to  “SDP Superimposition
>      >>>> Grouping framework” as the previous name wasn’t an accurate
>      >>>> description of the content. Therefore, we had to start from version 00.
>      >>>>
>      >>>> Thank you all for the comments/suggestions on the previous draft.
>      >>>> We have tried to address them in this version. Specifically, a
>      >>>> transparency attribute and an informative Section on relationship
>      >>>> with CLUE has been added.
>      >>>>
>      >>>> Comments/Questions/Feedbacks are welcome.
>      >>>>
>      >>>> Best Regards,
>      >>>> Rohit
>      >>>>
>      >>>> On 12/11/20, 12:41 PM, "[e-mail]"
>      >>>> <[e-mail]> wrote:
>      >>>>
>      >>>>
>      >>>>       A new version of I-D,
>      >>>> draft-abhishek-mmusic-superimposition-grouping-00.txt
>      >>>>       has been successfully submitted by Rohit Abhishek and posted to
>      >>>> the
>      >>>>       IETF repository.
>      >>>>
>      >>>>       Name:        draft-abhishek-mmusic-superimposition-grouping
>      >>>>       Revision:    00
>      >>>>       Title:        SDP Superimposition Grouping framework
>      >>>>       Document date:    2020-12-11
>      >>>>       Group:        Individual Submission
>      >>>>       Pages:        8
>      >>>>       URL:
>      >>>> https://www.ietf.org/archive/id/draft-abhishek-mmusic-superimpositio
>      >>>> n-grouping-00.txt
>      >>>>
>      >>>>       Status:
>      >>>> https://datatracker.ietf.org/doc/draft-abhishek-mmusic-superimpositi
>      >>>> on-grouping/
>      >>>>
>      >>>>       Htmlized:
>      >>>> https://datatracker.ietf.org/doc/html/draft-abhishek-mmusic-superimp
>      >>>> osition-grouping
>      >>>>
>      >>>>       Htmlized:
>      >>>> https://tools.ietf.org/html/draft-abhishek-mmusic-superimposition-gr
>      >>>> ouping-00
>      >>>>
>      >>>>
>      >>>>
>      >>>>       Abstract:
>      >>>>          This document defines semantics that allow for signaling a
>      >>>> new SDP
>      >>>>          group "S" for superimposed media in an SDP session.  The "S"
>      >>>>          attribute can be used by the application to relate all the
>      >>>>          superimposed media streams enabling them to be added as an
>      >>>> overlay on
>      >>>>          top of any media stream.  The superimposition grouping
>      >>>> semantics is
>      >>>>          required, if the media data is separate and transported via
>      >>>> different
>      >>>>          sessions.
>      >>>>
>      >>>>
>      >>>>
>      >>>>
>      >>>>       Please note that it may take a couple of minutes from the time
>      >>>> of submission
>      >>>>       until the htmlized version and diff are available at
>      >>>> tools.ietf.org.
>      >>>>
>      >>>>       The IETF Secretariat
>      >>>>
>      >>>>
>      >>>> _______________________________________________
>      >>>> mmusic mailing list
>      >>>> [e-mail]
>      >>>> https://www.ietf.org/mailman/listinfo/mmusic
>      >>>> _______________________________________________
>      >>>> mmusic mailing list
>      >>>> [e-mail]
>      >>>> https://www.ietf.org/mailman/listinfo/mmusic
>      >>>>
>      >>>
>      >>> _______________________________________________
>      >>> mmusic mailing list
>      >>> [e-mail]
>      >>> https://www.ietf.org/mailman/listinfo/mmusic
>      >>
>      >> _______________________________________________
>      >> mmusic mailing list
>      >> [e-mail]
>      >> https://www.ietf.org/mailman/listinfo/mmusic
>      >
>      > _______________________________________________
>      > mmusic mailing list
>      > [e-mail]
>      > https://www.ietf.org/mailman/listinfo/mmusic
>      > _______________________________________________
>      > mmusic mailing list
>      > [e-mail]
>      > https://www.ietf.org/mailman/listinfo/mmusic
>      >
> 
>      _______________________________________________
>      mmusic mailing list
>      [e-mail]
>      https://www.ietf.org/mailman/listinfo/mmusic
> 
> _______________________________________________
> mmusic mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/mmusic
>