Re: [clue] a few issues on data model draft

Hi Roni,

While what you say seems to be applicable here to CLUE, a couple of things spring to mind:

- it's not clear that we'll be declaring SSRC values in SDP as webrtc dies, so being able to specify imageattr there might not be of use

- even if we were, there's no obvious defined SSRC <-> media capture mapping being considered. Specifically, at offer-time the provider wouldn't know which SSRCs were to be used for which captures and so wouldn't be able to determine which imageattr values to assign with which SSRC values.

I can understand how what you describe works fine for webrtc - did you have a scheme in mind for tackling the above entwined issues (especially the second one) ?

Regards,

Andy

On 1 Aug 2012, at 02:25, "Roni Even" <ron.even.tlv@gmail.com> wrote:

> Andy,
> The image attribute provides a better solution  to allow the consumer to ask for a preferred image size which is not only the aspect ratio but the resolution and pixel aspect ratio.  It can be specified per SSRC http://tools.ietf.org/html/draft-lennox-mmusic-sdp-source-selection-03
> The consensus in RTCweb was to use this solution  based on http://tools.ietf.org/html/draft-alvestrand-rtcweb-resolution-00
>  
> Roni
>  
> From: Andy Pepperell [mailto:apeppere@gmail.com] 
> Sent: 01 August, 2012 1:41 AM
> To: Allyn Romanow (allyn)
> Cc: Espen Berger (espeberg); Roni Even; CLUE
> Subject: Re: [clue] a few issues on data model draft
>  
> >>>Currently it is not possible to tell whether a remote video source is natively 4:3 or 16:9, for instance.
> >>>If we have this, there might usefully be a "PREFERRED-ASPECT-RATIO" in the consumer's stream selection message as part of the  (e.g. in "<stream-description> / <encoding>").
> 
> >>Roni: RE: No need, first of all it does not work with switched capture that may have different aspect ratios. Also this is available in the SDP as well as a way for receiver to ask for specific ratio using the SDP image attribute.
> 
> >AR – I’ll let Andy respond to this
> 
> Many apologies for not doing so until now...
> 
> Roni, I think you're right in that at least the native aspect ratio as potentially expressed in a provider's capture advertisement wouldn't work for the switched capture case (or at least wouldn't work all the time - for the endpoint case where the switched capture was a dynamic choice between one of the "real" cameras' image then it would be possible (and straightforward) to attach the same native aspect ratio to the switched capture as was applicable to the non-switched captures). Even if it were impossible to attach a "native aspect ratio" to the switched case, to my mind this doesn't necessarily take away from the validity in this scenario of the consumer supplying a "preferred aspect ratio" in its stream selection message.
> 
> re: use of "imageattr" in the SDP, it's definitely pertinent here; potentially this has some issues in relation to CLUE:
> 
> - it would have to be applied on a per m line basis (excluding payload type differences) rather than per capture - within CLUE there would be a need (or at least use) of the aspect ratio specification being more fine-grained and separately specifiable per capture (this may depend in part on how we eventually choose to manage main vs slides video in terms of RTP session association though).
> 
> - by being in SDP, it's presumably the case that to correctly signal to the consumer when, say, a 4:3 presentation source had been connected would require an additional offer / answer cycle (and similarly if that source were to be disconnected and a new 16:9 source attached) - this seems slightly less lightweight than the equivalent operation would be if its scope were restricted to just the CLUE provider capture advertisement. As a side point, it also seems more heavyweight than SIP without the use of imageattr too, where presumably the only impact of connecting such a presentation source would be a BFCP exchange.
> 
> I'm happy to own up here to not having seen this attribute being present in devices' SDP, so if any of the above are inaccurate due to my ignorance on this, I apologise. It would certainly be interesting to get some sort of picture of the adoption of "imageattr" within the vendor community - I don't recall seeing this in use, but most of my exposure has been to systems not necessarily running recent software versions. Obviously if the already-defined "imageattr" does all we'd like within CLUE, then we should use this already-defined scheme.
> 
> Regards,
> 
> Andy
> 
> 
> On Sun, Jul 29, 2012 at 4:58 AM, Allyn Romanow (allyn) <allyn@cisco.com> wrote:
> Hi Espen,
> Thanks for your comments-
> Please see below-
> Also inline in response to your inline J
>  
> From: Espen Berger (espeberg) 
> Sent: Monday, July 23, 2012 8:27 AM
> To: Allyn Romanow (allyn); Roni Even; 'CLUE'
> 
> Subject: RE: [clue] a few issues on data model draft
>  
> I have two questions to the data model draft.
>  
> The first questions is related to naming differences between the CLUE framework and the CLUE data model. Some examples of differences. the framework uses Description and the data model uses capture-scene-text, the framework uses capture and the data model uses capture-description . Isn’t it natural that both documents share the same vocabulary?
>  
> AR – Christian also brought this up.   I believe that the documents will not have the same vocabulary – but the concepts are the same.     The framework was written first and it’s goal is to describe things as clearly as possible (we’re not done with that…). The data model is more formal and the language there is not casual. It was developed after the framework, so it sort of distils the ideas in the framework. 
>  
> If there is a discrepancy of meaning, that is an issue and we should definitely sort it out. But in terms of language, I think it’s fine to either leave it different or  change the framework at some point to be more in keeping with the data model.. but I think it’s a good idea to do that later when the data model is about agreed upon.                
>  
> I’ve put this on the list for discussion at the IETF meeting on Weds.                                                                                                                        
>  
> The second questions is related to the concept of doing a data model as the basis for both advertisement and configuration messages. Playing around with some XML examples messages it’s not obvious that the two messages types share a common data model. I do see similarities between the initial advertisement, the current active configuration and the advantages of be able to describe the state of a running CLUE system. Assume this will be sorted out when we look at the details of how to do configuration in CLUE.  
>  
> AR - Okay, this is interesting. Guess I’ll have to wait to see more specifically what issues arise.
>  
> Additional comments inline.
>  
> -Espen
>  
>  
> From: clue-bounces@ietf.org [mailto:clue-bounces@ietf.org] On Behalf Of Allyn Romanow (allyn)
> 
> Sent: 18. juli 2012 23:00
> To: Roni Even; 'CLUE'
> Subject: Re: [clue] a few issues on data model draft
>  
> Hi Roni,
> Sure, these are easy to add-
> thanks
>  
> From: Roni Even [mailto:ron.even.tlv@gmail.com]
> Sent: Wednesday, July 18, 2012 2:56 PM
> To: Allyn Romanow (allyn); 'CLUE'
> Subject: RE: [clue] a few issues on data model draft
>  
> Hi,
> Reviewed the draft, inline for your question (BTW: not all the new attributes here are in BOLD for example captureID)
>  
> Other comments
>  
> 1.       Need to add axis of capture to the media capture (now in the framework)
> 
> 2.       The capture scene text and language should be sets of (text, language), I think is not the current structure
> 
>  
> Thanks
> Roni
>  
> From: clue-bounces@ietf.org [mailto:clue-bounces@ietf.org] On Behalf Of Allyn Romanow (allyn)
> Sent: 18 July, 2012 3:22 AM
> To: CLUE (clue@ietf.org)
> Subject: [clue] a few issues on data model draft
>  
> Folks,
> 
> There are several issues related to the data model that we’d like us to consider. These are things that are not exactly the same as in the framework, to a lesser or greater extent. They were brought up during the data model presentation at the interim meeting. I wonder if we could work on resolving them?
> 
> List based on draft-romanow-clue-data-model-01
> 
> 1.  Any problem with using media-type for capture attribute as described in the draft?
> 
> RE: OK
> 
> 
> EBE> Supported, since its allow us to join the audio and video encodings into a single list of encodings.
> 2.  Any problem with using content-type for capture attribute as described in the draft?
> 
>                 RE: I did not see any difference from the framework in which case I am OK.
>                EBE> The framework use the ‘content’ name and the data model the ‘content-type’ name. Why not use the same name?  
> AR- as above
> 
> 3.  DERIVED for capture attribute- Should we have a variable called something like DERIVED, that takes various values, one of which is composed. That would mean that if DERIVED is not =composed, that the capture is not composed, and if it =composed then it is. I don’t have another descriptor to propose now, but there could be others either now or later.
> 
> Why this change from the framework? I think it is more general than composed Boolean, yet semantically similar.
> 
> This does not prevent us from having qualifiers to describe composed, to help  distinguish between different composed captures, as has  been discussed. Also, it needs to be possible to have multiple values for “DERIVED”, that is the values should not be mutually exclusive.
> 
>                 RE:  I am OK if this is for future extension but am not sure about the structure  the high level derived which has Compose and future something (not sure what). The composed has a value (true, false) and child elements for extensions . Is this the proposed structure?
> EBE> I like the idea of describing the origin of the capture with an attribute that is more extensible than a Boolean. What about creating enumerated values for the two origins we are currently discussing. We have ‘composed’ (composed=true) meaning that the media has been changed in some way and original (composed=false).
> 4.  Leave switched as a Boolean as in framework. The proposal in the draft seems infeasible – we don’t want CLUE  to have to include all the ways in which products might want to implement switching. It would be better to make this more general, but we have no suggestion for it at present- it does not really match the meaning of a derived capture.
> 
>                 RE: I think that if we have the extension for composed as I outlined above we can have similar here.
> 
> 5.  As part of the capture description, should we have a parameter NATIVE-ASPECT-RATIO, which indicates the aspect ratio for the capture if there is one, such as 4:3 for a video camera? The reason is that it can help rendering.
> 
> 
> 
> Currently it is not possible to tell whether a remote video source is natively 4:3 or 16:9, for instance. If we have this, there might usefully be a "PREFERRED-ASPECT-RATIO" in the consumer's stream selection message as part of the  (e.g. in "<stream-description> / <encoding>").
>                 RE: No need, first of all it does not work with switched capture that may have different aspect ratios. Also this is available in the SDP as well as a way for receiver to ask for specific ratio using the SDP image attribute.
> EBE> The image attribute (RFC6236) document talks about image resolution and picture aspect ratio, and how they relates to codec specific parameters like h264 max-fs and max-mbps.  An attribute can have a single value, a list of values or a range.
> 6.   Multiplex ID – Value sent by consumer in its stream selection message with the encoding id and capture id. This is the mechanism by which the consumer tells the provider which multiplex id to generate that stream with, so that the consumer can demultiplex the stream correctly when receiving it.
> 
> This is the link between RTP and CLUE as discussed in draft-lennox-clue-rtp-usage.
>      Think should probably be re-named.
> 
>                RE: I do not see the need the mapping is to the capture ID which is a numeric value. See http://tools.ietf.org/html/draft-even-clue-rtp-mapping-03
> 
>            7.  AUDIO-RENDERING-ID –  If the notion of an audio rendering tag is
> 
>      adopted, this would be needed. It is an ID provided by the consumer to the
> 
>      provider. The provider uses the tag to associate an audio stream with a video
> 
>     stream.
> 
>                 RE; suggest to take this out at the moment until we agree if needed and how to map.
> EBE> I assume that matching RTCP SDES CNAME is still the preferred mechanisms for syncing audio and video for lip-sync (as defined in RFC3550). When we discussed audio tags last interim it as was presented as a tool for improving directional audio.
>  
> 
> Clearly this doesn’t cover all the data model issues, but it’s a start.
> 
>  
> 
> Thanks,
> 
> Andy and Allyn
> 
>  
>  
> 
> _______________________________________________
> clue mailing list
> clue@ietf.org
> https://www.ietf.org/mailman/listinfo/clue
> 
>  