Re: [AVT] spatial-resolution parameter for RFC 3984bis

[Randell Jesup <[e-mail]>]

"Ingemar Johansson S" <[e-mail]> writes:
>To answer Roni's and Ye-Kui's comments I believe that the image attribute
>can be used for 3. However, looking in perspective I am not longer sure
>that the possibility to specify a framerate range makes sense as it does
>not connect  too wellto reality when it comes to e.g H.264 which talks
>about macroblocks per second. The conclusion is then that the framerate
>(or MBs) is determined from the fmtp line.
>This means that if the imageattr specifies a large offered range the frame
>rate is limited by what is possible for the given preferred resolution. 
>
>As an example, consider H.263 with the fmtp line
> CIF=4;QCIF=3;SQCIF=2;
>the imageattr may then be specified as
>a=imageattr:97 [x=[128:8:352],y=[96:8:288],par=[1.2,1.3]]
>If the receiver prefers the resolution 344x288 (slightly less than CIF) the maximum frame rate would be 30/(1.0001*4) ~ 7.5Hz. 
>If the prefered resolution is 160x128 (slightly less than QCIF) then the max frame rate is 30/(1.0001*3) Hz and so on.

So far as I know (and I didn't recheck), H.263 only allows for resolutions
specified in the fmtp (though QCIF and CIF are I think required).  If you
want 344x288, you need to add a CUSTOM resolution to the fmtp line.  And
the mpi must be an integer in the fmtp.  Now, if both sides use this, then
those restrictions could be relaxed.

>For the H.264 case the frame rate depending of the preferred image size
>would be limited according to the maximum framerate repending on the max
>MBs and the resolution given in table A-6 in the H.264 spec. For instance
>for level 2 an image size of 344x280 which is less than CIF but larger
>than QVGA would yield a max frame rate of 30Hz.

My suggestion would be for this to express preferences within the normal
negotiated resolution and framerate space.  i.e for 264 the level and
max-fs and max-mbps would determine the *maximum* frame and framerate
supported, but this would be used to decide what to actually send.

See my original email below -- look for:
    "I would suggest instead (and did in the thread back in March)"

>
>Would this distincion between framerate (or MBs) and preferred image size work?
>
>/Ingemar   
>
>
>
>> -----Original Message-----
>> From: Even, Roni [mailto:[e-mail]] 
>> Sent: den 28 juli 2008 11:32
>> To: [e-mail]; Ingemar Johansson S; [e-mail]
>> Cc: [e-mail]
>> Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis
>> 
>> Hi,
>> I think that we have to be clear about the usage of the parameter
>> 
>> 1. Announce sending of the specific resolution - just 
>> resolution 2. Request a specific resolution - Receiver wants 
>> a specific resolution 3. Negotiate available resolutions - 
>> The offer says, I can do CIF, QCIF, 1080P,... here it may 
>> dependent in H.263 and H.261 on the frame rate while in H.264 
>> the limit is in the macroblocks per second which will specify 
>> the maximum frame rate. The reason for this offer is because 
>> it offers the operation points that the offerer can have. The 
>> level parameter in H.263 and H.264 gives a maximum but does 
>> not imply support for any resolution that the answer would 
>> like to receive that falls within the range of the level.
>> 
>> I think the img attribute address 1 and 2, I am not sure about 3
>> 
>> Roni Even
>> 
>> > -----Original Message-----
>> > From: [e-mail] [mailto:[e-mail]]
>> > Sent: Monday, July 28, 2008 1:11 PM
>> > To: [e-mail]; [e-mail]
>> > Cc: Even, Roni; [e-mail]
>> > Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis
>> > 
>> > 
>> > Interesting! I just wrote the following email, planned to 
>> send to the 
>> > MMUSIC mailing list. Now this comes only to AVT. Feel free 
>> to forward 
>> > it to MMUSIC (don't know whether it is allowed to cross posting) or 
>> > let me to send that, if needed.
>> > 
>> > 
>> > After a reading of this draft (draft-johansson-mmusic-image- 
>> > attributes), I think the sprop-spatial-resolution parameter in the 
>> > RFC3984bis can go. It is completely covered by imageattr.
>> > 
>> > A couple of comments to draft-johansson-mmusic-image-attributes.
>> > 
>> > - It would be helpful to introduce the methods in payload 
>> format specs 
>> > for H.261, H.263 and H.264. In particular, for the one I am 
>> familiar 
>> > with (i.e. H.264), I am not sure how exactly image size 
>> together with 
>> > frame rate is negotiated.
>> > 
>> > - Re the following paragraph, I am not sure which parameters are 
>> > "similar features" and MUST NOT be specified in imageattr. 
>> Basically,
>> > H.264 sequence parameter set (SPS)may contain all 
>> information relating 
>> > to what may be included in imageattr according to the curent draft. 
>> > For example, the VUI part of SPS may even contain frame 
>> rate related 
>> > synatx elements.
>> >    o  Conflict with sprop-parameter-set: H.264 defines the sprop-
>> >       parameter-set which may define features that are 
>> defined in this
>> >       draft.  To avoid conflicts, sessions that use the imageattr
>> >       attribute MUST NOT specify similar features in 
>> sprop-parameter-
>> >       set.
>> > 
>> > BR, YK
>> > 
>> > >-----Original Message-----
>> > >From: ext Ingemar Johansson S
>> > >[mailto:[e-mail]]
>> > >Sent: Monday, July 28, 2008 1:04 PM
>> > >To: [e-mail]
>> > >Cc: Wang Ye-Kui (Nokia-NRC/Tampere); Even, Roni; Randell Jesup
>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis
>> > >
>> > >Hi
>> > >
>> > >
>> > >Given the discussion below I really wonder if it is not a better 
>> > >alternative to specifiy preferred image sizes using a dedicated 
>> > >(generic) SDP attribute.
>> > >Even though this is solved in some way in 3984bis we still have to 
>> > >provide with a solution for MPEG-4 or H.263 later on.
>> > >I drafted a proposed solution for an "a=imageattr" attribute some 
>> > >time ago that may be useful
>> > >http://tools.ietf.org/id/draft-johansson-mmusic-image-attributes-
>> > 01.txt
>> > >The draft provides with a solution that may work but lacks a 
>> > >requirement spec to outline exactly what the attribute 
>> aims for. Also 
>> > >extra work is needed to avoid conflicts with related 
>> payload format 
>> > >parameters.
>> > >
>> > >Regards
>> > >Ingemar
>> > >
>> > >
>> > >=============================
>> > >To: <Ye-Kui.Wang at nokia.com>
>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis
>> > >From: Randell Jesup <rjesup at wgate.com>
>> > >Date: Mon, 14 Jul 2008 09:35:18 -0400
>> > >Cc: roni.even at polycom.co.il, avt at ietf.org
>> > >
>> > ><Ye-Kui.Wang at nokia.com> writes:
>> > >>The resolution must be restricted by the level part of
>> > >>profile-level-id and max-fs. Therefore, if a configuration is agreed
>> > >>(or agreeable), then the answerer must request a resolution under
>> > >>the constraint, and the offerer must accept the request as long as
>> > >>it meets the restriction.  This is similar as sprop-parameter-sets,
>> > >>as long as it is complaint with the agreed (or agreeable)
>> > >>configuration, the other side must accept it.
>> > >
>> > >I would suggest instead (and did in the thread back in March): that
>> > >as a receiver property, it is the *preferred* maximum frame size.
>> > >The receiver is required to decode and handle any size up to the size
>> > >implied by the level (and max-fs if given), but the receiver prefers
>> > >this size (for example, it may support 1920x1080, but only have an
>> > >LCD display that can display 1200x720, so there's no *need* to send
>> > >it anything higher (and the encoder may prefer to avoid encoding more
>> > >macroblocks, especially in an MCU that shares resources)).  In other
>> > >cases, the sender may have the stream already encoded in different
>> > >resolutions, and this can help select the best one to send to save
>> > >encoding (and bandwidth) resources.
>> > >
>> > >There is no requirement that the encoder scale, but if it wants to it
>> > >can.
>> > >
>> > >It also (if I remember) provides aspect-ratio information.
>> > >
>> > >>Would it work this way?
>> > >>
>> > >>BR, YK
>> > >>
>> > >>>-----Original Message-----
>> > >>>From: ext Even, Roni [mailto:roni.even at polycom.co.il]
>> > >>>Sent: 14 July, 2008 10:38
>> > >>>To: Wang Ye-Kui (Nokia-NRC/Tampere); avt at ietf.org
>> > >>>Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis
>> > >>>
>> > >>>YK,
>> > >>>Just to clarify,
>> > >>>The text on the receiver side says that this is the requested 
>> > >>>resolution, it does not address issue of negotiation, 
>> what will the 
>> > >>>sender do if it cannot send this resolution, should it 
>> reject the 
>> > >>>stream?.
>> > >>>If this is for negotiation do you except to do a new 
>> offer answer 
>> > >>>if the receiver wants a new resolution?
>> > >>>
>> > >>>Roni
>> > >>>
>> > >>>> -----Original Message-----
>> > >>>> From: avt-bounces at ietf.org [mailto:avt-bounces at 
>> ietf.org] On 
>> > >>>> Behalf Of Ye-Kui.Wang at nokia.com
>> > >>>> Sent: Sunday, July 13, 2008 7:57 PM
>> > >>>> To: avt at ietf.org
>> > >>>> Subject: [AVT] spatial-resolution parameter for RFC 3984bis
>> > >>>>
>> > >>>>
>> > >>>> Concluded from the discussions in the email threads listed
>> > >below, I
>> > >>>> added a new parameter, spatial-resolution, to the RFC
>> > >3984bis draft
>> > >>>> (which will be submitted by tomorrow):
>> > >>>>
>> > >>>> http://www.ietf.org/mail-archive/web/avt/current/msg09425.html
>> > >>>> http://www.ietf.org/mail-archive/web/avt/current/msg09650.html
>> > >>>>
>> > >>>> The parameter is specified as follows (basically as Randell 
>> > >>>> suggested in 
>> > >>>> 
>> http://www.ietf.org/mail-archive/web/avt/current/msg09429.html).
>> > >>>>
>> > >>>> spatial-resolution:
>> > >>>> This parameter MAY be used to indicate the maximum spatial 
>> > >>>> resolution of a NAL unit stream or the preferred spatial
>> > >resolution
>> > >>>> of a receiver.
>> > >>>> The value is a base16 [6] (hexadecimal) representation of the
>> > width
>> > >>>and
>> > >>>> height of the spatial resolution, in pixels, separated 
>> by a comma.
>> > >>>>
>> > >>>> Usage in SDP offer/answer:
>> > >>>> - When "a=sendrecv" or no direction attribute is used, 
>> > >>>> "spatial-resolution" indicates both the maximum spatial 
>> > >>>> resolution of the stream sent by the declaring entity and the
>> > >preferred spatial
>> > >>>> resolution for receiving a stream.
>> > >>>> - When "a=sendonly", "spatial-resolution" indicates the maximum
>> > >>>spatial
>> > >>>> resolution of the outgoing stream.
>> > >>>> - When "a=recvonly", "spatial-resolution" indicates 
>> the preferred 
>> > >>>> spatial resolution for receiving a stream.
>> > >>>>
>> > >>>> Comments are welcome.
>> > >>>>
>> > >>>> BR, YK
>> > >>>> _______________________________________________
>> > >>>> Audio/Video Transport Working Group avt at ietf.org 
>> > >>>> https://www.ietf.org/mailman/listinfo/avt
>> > >>>
>> > >>_______________________________________________
>> > >>Audio/Video Transport Working Group
>> > >>avt at ietf.org
>> > >>https://www.ietf.org/mailman/listinfo/avt
>> > >
>> > >
>> > >--
>> > >Randell Jesup, Worldgate (developers of the Ojo 
>> videophone), ex-Amiga 
>> > >OS team rjesup at wgate.com "The fetters imposed on 
>> liberty at home 
>> > >have ever been forged out of the weapons provided for 
>> defence against 
>> > >real, pretended, or imaginary dangers from abroad."
>> > >		- James Madison, 4th US president (1751-1836)
>> > >
>> > >*******************************************
>> > >Ingemar Johansson
>> > >Senior Research Engineer, IETF "nethead"
>> > >EAB/TVP - Multimedia Technologies
>> > >Ericsson Research Ericsson AB
>> > >Box 920 S-971 28 Luleå, Sweden
>> > >Tel: [phone]
>> > >ECN: 850-43042
>> > >ECC: 850-43074
>> > >Mobile: [phone]
>> > >*******************************************
>> > >
>> 


-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[e-mail]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www.ietf.org/mailman/listinfo/avt