Re: [MMUSIC] Remarks from today

Hi Ted, Jonathan, all,

I see your point. It depends on the parameters that are chosen for the 3D stream, and how the legacy device responds to these parameters. The parameters are frame width, frame height and frame rate.

I think the main advantage of frame packing is that the 3D video can be carried as a regular 2D video stream. Therefore, we can distinguish the following cases:

1.       The 3D frame packed video is transmitted through a 2D video stream with fixed (prescribed) resolution/frame rate.

2.       The 3D frame packed video is transmitted through a 2D video stream with flexible resolution/frame rate.

In case the 3D frame packed video is transmitted through a 2D video stream with a fixed resolution/frame rate, the streaming server has to obey the restrictions of the 2D streaming system. For example, if the 2D streaming system has 1280x720 pixel^2 frames, and a frequency of 60Hz, the 3D video stream can have a resolution of 640x720 pixel^2, 60 Hz in case of side-by-side packing, or 1280x720 pixel^2, 30 Hz in case of frame sequential packing. In either case, there is no problem for a legacy 2D device receiving and displaying the stream, although it will not look good to the spectator.

If the 3D frame packed video is transmitted through a 2D video stream with flexible resolution/frame rate, there is more freedom in choosing the resolution/frame rate of the 3D stream too. The 3D streaming server could choose a higher resolution or frame rate of the 2D video stream, such that the corresponding 3D video has better quality. For example, if a resolution of 1280x720 pixel^2, 60 Hz is required for the 3D stream, the server could give the carrying 2D stream a resolution of 2560x720 pixel^2, 60Hz or 1280x720 pixel^2, 120Hz.

In either case, the legacy 2D receiver needs to understand the parameters of the carrying 2D video stream. It will go wrong when the parameters of the carrying 2D stream do not match the legacy 2D device's capabilities. But this is normal in the 2D case too; for example, if the system has fixed 1280x720 pixel^2, 60Hz, the server will have a problem attempting to the stream with any other resolution or frame rate. I think this issue is independent on whether the signal contains frame packed 3D video or not.

With a flexible resolution/frame rate, problems can still occur, for example when the server trespasses some upper limits in what the system can process. But this, too, I see as independent from the frame packing.

To come back to your question on shutter glasses: I think you mean a situation where a 3D video is frame sequential packed, with a frame rate that matches the frequency of the shutter glasses. Here the above two cases still hold: if the carrying 2D video stream has a fixed frame rate, the 3D video frame rate and shutter glass frequency can only be half the allowed 2D video stream frame rate. Otherwise the 3D stream cannot be carried in the 2D video stream. But if the restrictions on the 2D video stream are met, the 2D device will have no problem with displaying it.

What's your opinion?

Best regards,
Bert

From: Ted Hardie [mailto:ted.ietf@gmail.com]
Sent: 09 April 2011 01:51
To: Bert Greevenbosch
Cc: lennox@cs.columbia.edu; mmusic@ietf.org
Subject: Re: [MMUSIC] Remarks from today

On Thu, Apr 7, 2011 at 8:08 PM, Bert Greevenbosch <Bert.Greevenbosch@huawei.com<mailto:Bert.Greevenbosch@huawei.com>> wrote:
Hi Jonathan, all,

Interesting question! Here are some considerations:

For frame compatible 3D video, the backwards compatibility issue consists mainly in that the receiving legacy device will display the both L/R views, in the way they were packed into the video stream. This is not a very pleasing for the eye, but it should not lead to big technical problems.

Can you clarify this further, or provide a reference? I'm curious in particular about what you think a legacy device will use as a display framerate when it's getting signal originally intended for use with shutter glasses.

regards,

Ted

However, in the simulcast and 2D+aux transmitted as two streams case, the problem is bigger, as there are two "m=video ..." lines, and a choice needs to be made which video to show. In the simulcast case, any of the L- and R-streams can do. But in the 2D+aux case, it would be best if the receiver showed the 2D stream, rather than the auxiliary stream.

In case of SDP offer/answer, the problem is similar to the problem described in the SDP grouping spec (RFC 5888). Section 9.4.2 gives an overview on what will happen when the answerer does not understand the "group" attribute, in which case client behaviour is not clear. According to that text, the client might refuse the offer, in which case it is easy for the offerer to offer a 2D session instead. But if the client does not refuse the offer, it is up to the client to decide how to process the multiple video lines.

Related is the fact, that the "3DS" grouping semantics will not be understood by legacy devices. This is not a new problem, for example, RFC 4756 about Forward Error Correction (FEC) grouping, also describes the problem. In that RFC, there are listed two possible responses:
(1) The answerer returns an answer that ignores the grouping attribute.
(2) The answerer sends a refusal to the request (e.g., 488 Not acceptable here or 606 Not acceptable in SIP).

In the first case, the offerer understands "3DS" grouping is not supported and can send a new offer for only a 2D connection.
In the second case, the offerer can also try to send a new offer advertising for only a 2D connection, although it cannot be sure that the failure was caused by the "3DS" grouping beforehand.

RFC 5583, about the Media Decoding Dependency "MDD" grouping, lists similar behaviour for SDP offer/answer (section 6.1). Section 6.2 addresses the case in which a session is advertised using RTSP. and the legacy device does not understand "MDD" grouping. They differentiate the following cases:
(1) If at least one of the media types included in the SDP is within the receiver's capabilities, it selects among those candidates according to implementation specific criteria for setup, as usual.
(2) If none of the media types included in the SDP can be processed, then obviously no setup can occur.

This would mean, that in case of simulcast L/R streams the receiver will most likely display one of the two streams, which is OK.

But in the 2D+aux case, the receiver might either display the 2D stream or the auxiliary stream.

Best regards,
Bert

-----Original Message-----
From: lennox@cs.columbia.edu<mailto:lennox@cs.columbia.edu> [mailto:lennox@cs.columbia.edu<mailto:lennox@cs.columbia.edu>]
Sent: 01 April 2011 20:41
To: Bert Greevenbosch
Cc: mmusic@ietf.org<mailto:mmusic@ietf.org>
Subject: Re: [MMUSIC] Remarks from today
On Friday, April 1 2011, "Bert Greevenbosch" wrote to "mmusic@ietf.org<mailto:mmusic@ietf.org>" saying:

> Hi all,
>
> Thank you for your remarks during today's session. Here are the remarks I received concerning the 3D Format and Parallax Signalling drafts:
>
[...]
>
> Please let me know if I missed some questions/comments, or if you have any new questions/comments on the drafts.

I had one other question I was going to ask, but the line had been closed.

What is the backward-compability story here?  It seems like a non-3D-aware
endpoint receiving this SDP message is going to have a lot of problems
understanding what it means.  (Remember that in SDP, unknown attributes are
ignored.)

Packed frames are obviously the most problematic case here, but even the
simulcast and auxiliary modes are potentially unappealing.

--
Jonathan Lennox
lennox@cs.columbia.edu<mailto:lennox@cs.columbia.edu>
_______________________________________________
mmusic mailing list
mmusic@ietf.org<mailto:mmusic@ietf.org>
https://www.ietf.org/mailman/listinfo/mmusic