[httpstreaming] Efficient manifest push (Re: FW: New Version Notification for draft-zong-httpstreaming-gap-analysis-01)

[Mark Watson <[e-mail]>]

Here is an idea sparked by Thomas' mention below of AJAX and RSS push services.

In these services the client establishes a long-lived HTTP connection to a server on which it sends a single request. The response comes back in chunks, over time, enabling the server to "push" new content as it becomes available. Generally caches and proxies are transparent to this, although I think it does not work with some older proxies which expect to receive the whole response from upstream before returning anything to the client. Perhaps these are all gone by now. But anyway, the "chunks" cannot really be cached as the proxy has no idea what they are.

In the case of a document, such as a manifest, which is being periodically updated, or a sequence of different files, one could very simply expose these semantics in a standard way, which would enable caches to do their thing.

For example, suppose we define a new MIME type, multipart/versions, where each part of the multipart MIME response is a different version of the originally requested resource. A smart cache receiving a request for this resource can cache the "parts" as they arrive, each replacing the previously cached version. It can serve multiple incoming persistent connections with one upstream persistent connection, providing scalability. It would be transparent to existing caches. Clients would indicate their support in the Accept header and clients which did not support this mode would just poll the resource in the usual way with conditional GET requests.

All that would be required from a standards perspective would be definition of the multipart/versions MIME type.

Maybe this is not new. I could easily imagine CDNs do this kind of thing internally already.

Similarly, one could imagine a multipart/sequence MIME type where the parts form a sequence of objects. The client requests an object and (if it indicates support of the multipart/sequence MIME type) gets back that object and subsequent ones in the sequence (with their file names). Again, smart caches could optimise for scalability, caching the parts as separate objects. This could help with delivery of segments in the live case, but again maintaining backwards-compatibility. Again, perhaps CDNs already do something like this internally.

I'm not proposing to progress these ideas, just thought they were interesting.

...Mark




On Oct 27, 2010, at 11:51 PM, Thomas Stockhammer wrote:

> Ning,
> 
> thanks ....
> 
> I recognized that you only replied to some of my comments.
> Does this mean that you agree/disagree with the remaining ones?
> 
> Inline some more with [T] ... [\T]
> 
> Thomas
> 
>> - I am not sure I understand the term "is encrypted into files"
>> [ZN]: I mean "use file with media container" here.
> 
> [T]  I do not understand this either! [\T]
> 
>> - What do you mean "normal text file"?
>> [ZN]: traditional web page (e.g. html file).
> 
> [T] we should be much more careful with terminology [\T]
> 
>> - The intelligence in the Adaptive Streaming over HTTP is almost
>> exclusively in the client, there is no negotiation
>> [ZN]: Sorry for confusion, "negotiation" should be "massage exchange".
> 
> [T]
> First I hope this is a typo, otherwise I get more curious ...!
> Secondly, I am still not clear what needs to be done beyond regular  
> http connections
> [\T]
> 
>> 5.2)
>> - It is not correct that the 3GPP MPD needs to be updated even for
>> live. If you use a template mode, the MPD stays static until some
>> "unforeseen" event occurs. Client and Content Preparation have agreed
>> on rules to construct URIs.
>> - If necessary, the MPD update happens asynchronously to the media
>> decoding, so this is not considered to be a problem.
>> [ZN]: I didn't intend to state that pull model doesn't work. My  
>> point is,
>> why not investigating the possible usage of push model in certain  
>> cases
>> without experiencing the above mentioned "unforeseen" event or  
>> asynchronous
>> updates?
> 
> [T]
> "Push" is a very very broad term. In Web applications you can for  
> example use AJAX or RSS/ATOM like techniques for push-like updates. If  
> you use conditional GET for regular polling, this is very efficient.  
> The MPD updates in 3GPP work in a similar manner. If you use polling,  
> conditional GETs and templates, you are extremely efficient. We should  
> really understand what we mean by push model? HTTP-based delivery is  
> rich and provides many successfully deployed options.
> 
> Should you really refer to something completely different such as IP  
> multicast, then I would feel very very uncomfortable.
> [\T]
> 
>> - There is for sure mechanisms to deliver important packets more
>> reliably in HTTP - you just request it earlier. In anticipation of
>> switching a smart client may also prepare such data. The client is
>> intelligent.
>> [ZN]: Well, I think this startup issue doesn't like the pre-fetch  
>> which is
>> of course still valuable to improve playback. IMO, it is hard to  
>> predict
>> which channel the user will switch to in the next moment, hence it  
>> is not
>> reasonable to request important packets for other channels. Did I
>> misunderstand you?
> 
> [T]
> I would not be worried to have the MPD and the initialization segment  
> of the two neighboring channels ready in my device. Again, the client  
> can be very smart, especially as it does have access to all the  
> information.
> 
> In general, I do not disagree that we can create more detailed use  
> cases for environments in which we envision that HTTP streaming will  
> be used. This may include live multi-channel environments. However, we  
> should not conclude per se that the existing technologies do have a  
> problem.
> [\T]
> 
>> - 3GPP defines QoE metrics that can reused also for HTTP Streaming.
>> there will also be efforts in MPEG in including QoE.
>> [ZN]: I am not proposing to focus on defining QoE metrics, but  
>> looking on
>> the protocols to report such metrics, like RTCP. We will support the  
>> work in
>> 3GPP/MPEG and cooperate with them to see how to capsulate the  
>> metrics in a
>> series of messages.
> 
> [T]
> What do you mean with "capsulate"?
> Also, can you be more specific what metrics there are in RTCP that can  
> also be used in HTTP Streaming. I consider that anything dealing with  
> packet loss is irrelevant. I also do not see the relevance of sending  
> regular 5 seconds receiver reports as the content is static and  
> adaptation will not happen. Some reporting on Media Presentation level  
> may be sufficient, for example when the presentation has been completed.
> [\T]
> 
> 
> ---
> Dr. Thomas Stockhammer (CEO) || [e-mail] || phone +49 89  
> 978980 02 || cell [phone] || http://www.nomor-research.com
> Nomor Research GmbH  -  Sitz der Gesellschaft: München -  
> Registergericht: München, HRB 165856 – Umsatzsteuer-ID: DE238047637 -  
> Geschäftsführer: Dr. Thomas Stockhammer, Dr. Ingo Viering.
> 
> 
> 
> 
> _______________________________________________
> httpstreaming mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/httpstreaming
>