Re: [MMUSIC] The 3D drafts

[[e-mail]]

Hi Bert,

Thanks for answering.

The usage of the BUNDLE mechanism to handle the transmission of video and
auxiliary maps in a single stream is an interesting idea. I have a few
comments about it:

1) Media Structure: The base assumption for BUNDLE is that several media
streams are sent over a single RTP session, and sharing the same 5-tuple.
These media streams can be differentiated from one another by their
payload type and SSRC. However, a stream containing 2D video and an
auxiliary map doesn't follow the same structure: the auxiliary map is
included as metadata within the 2D video stream. I don't know whether that
could be a problem. At the very least, if BUNDLE and 3dFormat are used
together as you suggest, the specification should state clearly that this
is a deviation from the normal behavior of BUNDLE.

2) When to use BUNDLE: Should the BUNDLE mechanism be used only in the
specific scenario of 2D video and auxiliary streams with different
encodings, or would it apply to all scenarios with 2D+aux in a single
stream? In the latter case, BUNDLE would effectively replace the formats
"CD", "LD", "CP" and "LP", as defined in the current draft. This makes for
more complex and verbose session descriptions, but is probably a more
consistent approach than supporting the syntax with BUNDLE and the one
without.

3) Your example SDP: You don't really need that many streams and groups
when combining 3dFormat and BUNDLE. A more compact way to describe the
same offer would be to use 2 streams, offering 2 different payload types
for the auxiliary map. Incidentally, this is also more efficient than the
mechanism I suggested in my original example:

             a=group:3DS 1 2
             a=group:BUNDLE 1 2
             m=video 49170 RTP/AVP 99
             a=rtpmap:99 H264/90000
             a=3dFormat:SC C
             a=mid:1
             m=video 49170 RTP/AVP 100 101
             a=rtpmap:100 H263/90000
             a=rtpmap:101 H264/90000
             a=3dFormat:SC D
             a=mid:2

Best regards,
Pedro

> Hi Pedro,
>
> As promised, here the answer to your second e-mail.
>
> The current simulcast solution in the draft indeed assumes a single codec,
> which internally contains both a 2D and an auxiliary stream.
>
> But my understanding is that your main concern is the simulcast of two
> streams (2D and aux), with different codecs. Your proposed solution would
> indeed resolve that problem.
>
> Now that we have the discussions on RTP multiplexing, especially the
> "BUNDLE" draft
> (http://datatracker.ietf.org/doc/draft-holmberg-mmusic-sdp-bundle-negotiation),
> I was wondering whether we could somehow use that mechanism to resolve the
> problem. In that case, your original example
>
>              m=video 49170 RTP/AVP 99 100
>              a=3dFormat:2DA CD
>              a=rtpmap:99 H264/90000
>              a=3daux: 99 H263/90000 aux-fmtp:(...)
>              a=rtpmap:100 H264/90000
>              a=3daux: 100 H264/90000 aux-fmtp:(...)
>
> would become as follows:
>
>              a=group:3DS 1 2
>              a=group:BUNDLE 1 2
>              m=video 49170 RTP/AVP 99
>              a=rtpmap:99 H264/90000
>              a=3dFormat:SC C
>              a=mid:1
>              m=video 49170 RTP/AVP 100
>              a=rtpmap:100 H263/90000
>              a=3dFormat:SC D
>              a=mid:2
>              a=group:3DS 3 4
>              a=group:BUNDLE 3 4
>              m=video 49170 RTP/AVP 99
>              a=rtpmap:99 H264/90000
>              a=3dFormat:SC C
>              a=mid:3
>              m=video 49170 RTP/AVP 100
>              a=rtpmap:100 H264/90000
>              a=3dFormat:SC D
>              a=mid:4
>
> This means, that we use the separate streams mechanism + the "BUNDLE"
> mechanism to achieve our goal. It this is maybe a bit premature, as the
> "BUNDLE" draft is only a personal draft, but it seems to work and we would
> not need an extra mechanism in the 3D format draft.
>
> Admittedly, this solution also looks complex, especially because of the
> double grouping mechanism. However, it would not require to define a new
> "3daux" attribute and "aux-fmtp" semantics.
>
> What do you think?
>
> Best regards,
> Bert
>
>
> From: Pedro Capelastegui [mailto:[e-mail]]
> Sent: 10 February 2012 20:33
> To: Bert Greevenbosch; 'IETF - MMUSIC'
> Subject: RE: [MMUSIC] The 3D drafts
>
> Hi Bert,
>
> After taking another look at the "3d format" draft, I have found a few
> additional issues. They are summarized below:
>
>
> 1)      Encoding of auxiliary stream: In the "video plus auxiliary stream"
> schemes using a single stream (i.e. "LD", "LP",  "CD", and  "CP"), no
> encoding information is provided for the auxiliary stream.
>
> 2)      Ambiguous negotiation for scenarios with multiple 3D streams:
> There is no clear way for an offerer to indicate when multiple 3D streams
> (e.g. to show in multiple displays) are offered, as opposed to a single 3D
> stream with several possible configurations.
>
> 3)      Support for H264/MVC streams: The H264/MVC codec is very well
> suited for 3d video applications. It uses some encoding formats that are
> not supported by the current draft but could be added without much effort.
> I'll cover each topic in a separate mail. Here is some discussion on point
> 1), auxiliary stream encoding.
>
> Auxiliary video streams such as depth maps or parallax can be encoded,
> just like any other video stream. In the "Signal 3d format" draft, the
> encoding for these streams can be described and negotiated normally when
> they are transmitted as separate RTP streams. However, this is no longer
> true when depth or parallax maps are encapsulated with a regular view as a
> single RTP stream, as defined in ISO 23002-3. This corresponds to the
> formats "2DA CD", "2DA CP" , "2DA LD" and "2DA LP" in the draft.
>
> It could be assumed that, since no encoding information is provided for
> these streams, they can use the same encoding used for the video view they
> are encapsulated with. As an example, consider the following stream:
>
> m=video 49170 RTP/AVP 99
> a=rtpmap:99 H264/90000
> a=3dFormat:2DA CD
>
> The above lines describe a video stream representing a central view, with
> a depth map included in the same RTP stream as auxiliary data. The central
> view is encoded in H264, but the depth map encoding is undefined. We could
> solve this by stating in the draft that, when no encoding is specified for
> an auxiliary stream, it uses the same configuration as the same view.
> Thus, the depth map in the example would also use H264.
>
> I think this works well as a default assumption. However, it does not
> cover all possible scenarios, as the auxiliary stream CAN use a different
> format than the base view. In fact, that will likely be the best
> performing configuration, since depth maps have different properties than
> regular video streams, and codecs like H264 don't perform particularly
> well with them.
>
> To address this, we would need a way to express the media format for
> auxiliary streams. At the very least, this would involve including the
> information normally expressed in "a=rtpmap" and "a=fmtp" attributes.
>
> One possible way to do this would be to add the values of these attributes
> (for the auxiliary map) as optional parameters in the "a=3dformat"
> attribute. In the previous example, this could look as follows:
>
> m=video 49170 RTP/AVP 99
> a=rtpmap:99 H264/90000
> a=3dFormat:2DA CD aux-map:H264/90000 aux-fmtp:(...)
>
> A limitation of this approach is that it just describes a single encoding
> for the auxiliary map. Ideally, a solution should allow the offerer to
> suggest multiple configurations from which the answerer can choose one, as
> with any media stream in the offer/answer model. For this, we could define
> a new attribute, like "a=3daux", and include in it a payload type number,
> the value of "a=rtpmap" for the auxiliary stream, and the value of
> "a=fmtp" for that stream. It could look as follows:
>
> m=video 49170 RTP/AVP 99 100
> a=3dFormat:2DA CD
> a=rtpmap:99 H264/90000
> a=3daux: 99 H263/90000 aux-fmtp:(...)
> a=rtpmap:100 H264/90000
> a=3daux: 100 H264/90000 aux-fmtp:(...)
>
> In this example, the offerer is providing two possible configurations:
>
>
> -          PT 99:  Central view encoded in H.264, depth map encoded in
> H.263
>
> -          PT 100: Central view and depth map encoded in H.264
>
> In addition, there may be other SDP fields or attributes that have to be
> taken into account while signalling these auxiliary streams.
>
> Best regards,
> Pedro
>
>
> From: [e-mail] [mailto:[e-mail]] On Behalf
> Of Bert Greevenbosch
> Sent: Monday, February 06, 2012 1:32 AM
> To: IETF - MMUSIC
> Subject: [MMUSIC] The 3D drafts
>
> Hi all,
>
> I was wondering, if everybody is happy with the current versions of the 3D
> drafts, or are there still open issues?
> I look forward to your comments.
>
> Best regards,
> Bert
>
> http://datatracker.ietf.org/doc/draft-ietf-mmusic-parallax-attribute/
> http://datatracker.ietf.org/doc/draft-ietf-mmusic-signal-3d-format/
> ________________________________
>
> No se encontraron virus en este mensaje.
> Comprobado por AVG - www.avg.com<http://www.avg.com>
> Versión: 10.0.1424 / Base de datos de virus: 2112/4791 - Fecha de
> publicación: 02/05/12
>