Re: [MMUSIC] The 3D drafts

[Bert Greevenbosch <[e-mail]>]

Hi Pedro,

As promised, here the answer to your second e-mail.

The current simulcast solution in the draft indeed assumes a single codec, which internally contains both a 2D and an auxiliary stream.

But my understanding is that your main concern is the simulcast of two streams (2D and aux), with different codecs. Your proposed solution would indeed resolve that problem.

Now that we have the discussions on RTP multiplexing, especially the "BUNDLE" draft (http://datatracker.ietf.org/doc/draft-holmberg-mmusic-sdp-bundle-negotiation), I was wondering whether we could somehow use that mechanism to resolve the problem. In that case, your original example

             m=video 49170 RTP/AVP 99 100
             a=3dFormat:2DA CD
             a=rtpmap:99 H264/90000
             a=3daux: 99 H263/90000 aux-fmtp:(...)
             a=rtpmap:100 H264/90000
             a=3daux: 100 H264/90000 aux-fmtp:(...)

would become as follows:

             a=group:3DS 1 2
             a=group:BUNDLE 1 2
             m=video 49170 RTP/AVP 99
             a=rtpmap:99 H264/90000
             a=3dFormat:SC C
             a=mid:1
             m=video 49170 RTP/AVP 100
             a=rtpmap:100 H263/90000
             a=3dFormat:SC D
             a=mid:2
             a=group:3DS 3 4
             a=group:BUNDLE 3 4
             m=video 49170 RTP/AVP 99
             a=rtpmap:99 H264/90000
             a=3dFormat:SC C
             a=mid:3
             m=video 49170 RTP/AVP 100
             a=rtpmap:100 H264/90000
             a=3dFormat:SC D
             a=mid:4

This means, that we use the separate streams mechanism + the "BUNDLE" mechanism to achieve our goal. It this is maybe a bit premature, as the "BUNDLE" draft is only a personal draft, but it seems to work and we would not need an extra mechanism in the 3D format draft.

Admittedly, this solution also looks complex, especially because of the double grouping mechanism. However, it would not require to define a new "3daux" attribute and "aux-fmtp" semantics.

What do you think?

Best regards,
Bert


From: Pedro Capelastegui [mailto:[e-mail]]
Sent: 10 February 2012 20:33
To: Bert Greevenbosch; 'IETF - MMUSIC'
Subject: RE: [MMUSIC] The 3D drafts

Hi Bert,

After taking another look at the "3d format" draft, I have found a few additional issues. They are summarized below:


1)      Encoding of auxiliary stream: In the "video plus auxiliary stream" schemes using a single stream (i.e. "LD", "LP",  "CD", and  "CP"), no encoding information is provided for the auxiliary stream.

2)      Ambiguous negotiation for scenarios with multiple 3D streams: There is no clear way for an offerer to indicate when multiple 3D streams (e.g. to show in multiple displays) are offered, as opposed to a single 3D stream with several possible configurations.

3)      Support for H264/MVC streams: The H264/MVC codec is very well suited for 3d video applications. It uses some encoding formats that are not supported by the current draft but could be added without much effort.
I'll cover each topic in a separate mail. Here is some discussion on point 1), auxiliary stream encoding.

Auxiliary video streams such as depth maps or parallax can be encoded, just like any other video stream. In the "Signal 3d format" draft, the encoding for these streams can be described and negotiated normally when they are transmitted as separate RTP streams. However, this is no longer true when depth or parallax maps are encapsulated with a regular view as a single RTP stream, as defined in ISO 23002-3. This corresponds to the formats "2DA CD", "2DA CP" , "2DA LD" and "2DA LP" in the draft.

It could be assumed that, since no encoding information is provided for these streams, they can use the same encoding used for the video view they are encapsulated with. As an example, consider the following stream:

m=video 49170 RTP/AVP 99
a=rtpmap:99 H264/90000
a=3dFormat:2DA CD

The above lines describe a video stream representing a central view, with a depth map included in the same RTP stream as auxiliary data. The central view is encoded in H264, but the depth map encoding is undefined. We could solve this by stating in the draft that, when no encoding is specified for an auxiliary stream, it uses the same configuration as the same view. Thus, the depth map in the example would also use H264.

I think this works well as a default assumption. However, it does not cover all possible scenarios, as the auxiliary stream CAN use a different format than the base view. In fact, that will likely be the best performing configuration, since depth maps have different properties than regular video streams, and codecs like H264 don't perform particularly well with them.

To address this, we would need a way to express the media format for auxiliary streams. At the very least, this would involve including the information normally expressed in "a=rtpmap" and "a=fmtp" attributes.

One possible way to do this would be to add the values of these attributes (for the auxiliary map) as optional parameters in the "a=3dformat" attribute. In the previous example, this could look as follows:

m=video 49170 RTP/AVP 99
a=rtpmap:99 H264/90000
a=3dFormat:2DA CD aux-map:H264/90000 aux-fmtp:(...)

A limitation of this approach is that it just describes a single encoding for the auxiliary map. Ideally, a solution should allow the offerer to suggest multiple configurations from which the answerer can choose one, as with any media stream in the offer/answer model. For this, we could define a new attribute, like "a=3daux", and include in it a payload type number, the value of "a=rtpmap" for the auxiliary stream, and the value of "a=fmtp" for that stream. It could look as follows:

m=video 49170 RTP/AVP 99 100
a=3dFormat:2DA CD
a=rtpmap:99 H264/90000
a=3daux: 99 H263/90000 aux-fmtp:(...)
a=rtpmap:100 H264/90000
a=3daux: 100 H264/90000 aux-fmtp:(...)

In this example, the offerer is providing two possible configurations:


-          PT 99:  Central view encoded in H.264, depth map encoded in H.263

-          PT 100: Central view and depth map encoded in H.264

In addition, there may be other SDP fields or attributes that have to be taken into account while signalling these auxiliary streams.

Best regards,
Pedro


From: [e-mail] [mailto:[e-mail]] On Behalf Of Bert Greevenbosch
Sent: Monday, February 06, 2012 1:32 AM
To: IETF - MMUSIC
Subject: [MMUSIC] The 3D drafts

Hi all,

I was wondering, if everybody is happy with the current versions of the 3D drafts, or are there still open issues?
I look forward to your comments.

Best regards,
Bert

http://datatracker.ietf.org/doc/draft-ietf-mmusic-parallax-attribute/
http://datatracker.ietf.org/doc/draft-ietf-mmusic-signal-3d-format/
________________________________

No se encontraron virus en este mensaje.
Comprobado por AVG - www.avg.com<http://www.avg.com>
Versión: 10.0.1424 / Base de datos de virus: 2112/4791 - Fecha de publicación: 02/05/12