Re: [ipwave] Adaptive RESTful Real-time Live Streaming for Things (A-REaLiST)'

[Alexandre Petrescu <[e-mail]>]

Le 30/01/2019 à 23:01, Abhijan Bhattacharyya a écrit :
> [...] The drone uses some proprietary variation of H.264 encoded 
> video that is delivered over TCP/IP. There had been several
> instances of long video freezes. We investigated. It seemed that
> TCP's state machine became a bottleneck.

If TCP's state machine was a bottleneck, has one tried to use UDP
instead?

> Also, the drone needs to manage the process memory (which is not too
>  high) in order to buffer the contents in case of a lossy 
> transmission so that TCP's reliable delivery mechanism can function 
> properly.

Makes sense to have little memory on the drone because memory is heavy.

> It would not be really worthwhile to compare CoAP, which is a RESTful
> application layer protocol, with TCP/IP transport. Rather, our
> approach was a bit different as will be explained below.
> 
> />I understand the need to transfer video (First-Person View, FPV), 
> but/ />I / />do not understand how CoAP helps with video. /
> 
> CoAP has been traditionally designed for lightweight transfer of 
> small sensor data for constrained devices in LLN.

Does CoAP work on Ethernet?

Does CoAP work in an end-to-end manner or does it need protocol
conversion gateways?

I am asking because although a drone may carry just one computer, the
in-car receiver of the video is deep in the car network.  The gateway
that connects that car is end-to-end: it is a Router, it is IP-OBU.  Can
CoAP go through that router.

> So video may sound a bit wiered at first in the context of CoAP. 
> However, video is nothing but a time-series information which needs 
> to be rendered in real-time (and in strict real-time for FPV based 
> remote decision engines).

Video is very important in car networks.  Not only one stream, but
multiple streams.  There are at least 4 IP cameras in a small car.
There are other kinds of video-like streams, like lidar.

802.11-NGV can easily accomodate all of them, because it promisses more 
bandwidth than 802.11p.

> We found that CoAP has a unique feature that allows seamless
> switching of protocol semantics (incurring no additional cost on
> resources) between reliable and best-effort delivery while
> transmitting the information segments of the stream. So, depending on
> certain application driven criteria, you can classify some parts of
> your information as critical and some as non-critical. The critical
> segments are delivered reliably and the non-critical ones with
> best-effort. The "criticality" of an information segment is
> determined by the fact, whether that segment is essential to render
> the video at the receiver. A naive example: In a temporally
> compressed video stream, the I-frames may be considered as critical,
> while the p-frames and b-frames may be sent with best effort because
> the rendering engine essentially syncs with the I-frames.

The I-frames should be sent with a particular QoS characteristic on OCB.

OCB link will ensure the criticality that you need, not TCP.

The current text in the IPv6-over-OCB draft that may help I-frames is 
the following:>    The IPv6 packet transmitted on 802.11-OCB MUST be 
immediately
>    preceded by a Logical Link Control (LLC) header and an 802.11 header.
>    In the LLC header, and in accordance with the EtherType Protocol
>    Discrimination (EPD, see Appendix E), the value of the Type field
>    MUST be set to 0x86DD (IPv6).  In the 802.11 header, the value of the
>    Subtype sub-field in the Frame Control field MUST be set to 8 (i.e.
>    'QoS Data'); the value of the Traffic Identifier (TID) sub-field of
>    the QoS Control field of the 802.11 header MUST be set to binary 001
>    (i.e.  User Priority 'Background', QoS Access Category 'AC_BK').
---end-of-excerpt-from-draft.


> Moreover, our approach follows the tradition set by HTTP where video
>  is considered just like any other web-traffic and HTTP-streaming  on
>  TCP/IP (the de facto mechanism for OTT video delivery at this point)
>  streams video over RESTful API calls. Same way, considering the 
> requirements of LLN and constrained devices, we tried to extend CoAP
>  so that we have a unified RESTful mechanism which is equally good in
>  transferring small sensor information, control, telemetry and 
> actuation information, as well as stream real-time time-series 
> information like video. Being RESTful helps to seamlessly connect to
>  other services on the web as well.

I want to ask you: please use IPv6 for CoAP and RESTful.

> />Video over IPv6 over OCB works fine between vehicles.
>> 
>> In the future, with the hopeful development of 802.11-NGV (Next 
>> Generation Vehicular), I expect latencies in the order of hundreds
>>  of
>> 
>> micro-seconds, and 1Gbps bandwidths. I think video over IP, and 
>> lidar streams over IP, would take advantage of that./
> As you may notice from the above, the proposed solution is agnostic 
> of what you use at the lower layers. One thing that we kept in mind 
> while conceiving this solution is that, you may have a radio link 
> with a promised high bandwidth, but you may have unpredictable 
> intermittent connectivity too due to random impairment / obstructions
> . As per our experience, drones flying within WiFi zone in an indoor
> environment also suffer from such unpredictable link behaviour due to
> sporadic zero-radio zones. This overpowers the promise of bandwidth.
> We tried to design the solution keeping those practical problems in
> mind.

The solution looks interesting.

> />But such a multi-feature protocol is also claimed by Google with 
> QUIC
>> 
>> protocol. Why not QUIC?/
> 
> I am not sure whether QUIC can cater the kind of flexibility we 
> wanted at the application layer (please forgive my limited knowledge
>  on this). Also, we wanted to create a solution which should be based
>  on something that already has a strong foundation through 
> standardization and wide adoption (may not be particularly in drones
>  right now - but who knows!). Furthermore, while performing the 
> state-of-the-art studies we encountered this paper: 
> https://dl.acm.org/citation.cfm?id=3083175 ( Not so QUIC: A 
> Performance Study of DASH over QUIC ). But, I do agree that further 
> investigations is worth carrying out to check the performance over 
> QUIC. But, perhaps that should not disqualify the proposed solution 
> leveraging CoAP at this point.

I am not an expert of QUIC either.

I am not here to disqualify anything.  I appreciate the advantages of 
solution.

It would be good if CoAP trials run on IPv6, CoAP to not rely on 
protocol gateways, and CoAP to map to OCB QoS classes.

> />I can look at the draft, but I need to understand why CoAP?/
> 
> It becomes a bit of a race-condition. :) We have explained this "why"
> in section 2 of the draft 
> (https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-00#page-5)

Thank you, I clicked and looked over it quickly.  I may have some comments.

Then I searched for the keyword 'IPv6' in the draft.


 > But to look into the draft you keep the explanation as a
> pre-requisite. To address this deadlock, I tried to briefly explain 
> the idea above in this mail. If you find it satisfactory, then may I
>  please request you have a look at the link and scroll through the 
> entire draft subsequently. You may also please go through page# 70-81
> of this deck: 
> https://datatracker.ietf.org/meeting/103/materials/slides-103-core-consolidated-slides-07

If you add 'IPv6' considerations to it, then I will comment on it.

Alex

>
>
>
>
> 
. We presented this in Bangkok.
> 
> />(and yes, I do agree there are strong IP requirements for video in
>> drones, like multi-cameras, multiple laser, infrared, and others)
> /
> 
> Thank you. Probably there is an extensive scope to contribute.
> 
> *</Alex>*
> 
> *<Michael>*
> 
> />And SCTP should be easy to deploy too if there is losing frames is
>> preferred to winding up non-realtime.
> /
> 
> First of all, we did not find much standard practices using SCTP for
>  this kind of applications in recent implementations. Again, we are 
> addressing the problem from the application-layer with a view of a 
> generalized web infrastructure. Furthermore, if you may have noticed
>  in the previous texts explaining the idea, losing frame is not the 
> design concern. The concern is: That loss has to be an informed 
> decision controlled by the application-layer. Also, if you may please
> have the opportunity to go through the draft, we are essentially
> proposing a rudimentary mechanism to marry application intelligence
> with protocol semantics.
> 
> *</Michael>*
> 
> More views and comments will be more than welcome.
> 
> With Best Regards Abhijan Bhattacharyya Consultant / Scientist, 
> {Internet Protocols | 5G | Standardization}, TCS Research, Tata 
> Consultancy Services Building 1B,Ecospace Plot -  IIF/12 ,New Town, 
> Rajarhat, Kolkata - 700160,West Bengal India Ph:- [phone] 
> Cell:- [phone] | [phone] Mailto: 
> [e-mail] <mailto:[e-mail]> 
> Website: http://www.tcs.com 
> ____________________________________________ Experience certainty. IT
> Services Business Solutions Consulting 
> ____________________________________________
> 
> 
> -----"its" <[e-mail] <mailto:[e-mail]>> 
> wrote: -----
> 
>> To: [e-mail] <mailto:[e-mail]> From: Alexandre Petrescu 
>> Sent by: "its" Date: 01/24/2019 04:48PM Subject: Re: [ipwave] 
>> Adaptive RESTful Real-time Live Streaming for Things (A-REaLiST)'
>> 
>> "External email. Open with Caution"
>> 
>> 
>> Le 24/01/2019 à 05:07, Abhijan Bhattacharyya a écrit :
>>> Dear list, We submitted a draft in CoRE with a technical proposal
>>> for
>> efficient
>>> transfer of time-series stream and presented in IETF Bangkok.
>>> The immediate use case that we considered was to transfer First 
>>> Person
>> View
>>> (FPV) video from a remote drone to a control unit over IP based 
>>> infrastructure. We built it on CoAP leveraging some of CoAP's
>> unique
>>> features.
>> 
>> Hi,
>> 
>> Thank you for the message.
>> 
>> I have seen another drone from DLR that communicates with a van, 
>> but not using CoAP. It uses TCP/IP family of protocols.
>> 
>> I understand the need to transfer video (First-Person View, FPV), 
>> but I do not understand how CoAP helps with video.
>> 
>> Video over IPv6 over OCB works fine between vehicles.
>> 
>> In the future, with the hopeful development of 802.11-NGV (Next 
>> Generation Vehicular), I expect latencies in the order of hundreds
>>  of
>> 
>> micro-seconds, and 1Gbps bandwidths. I think video over IP, and 
>> lidar streams over IP, would take advantage of that.
>> 
>> Why CoAP?
>> 
>>> The ambition is to create a protocol stack that would be equally
>>>  good in sending small sporadic bytes (sensor data) as well
>> as
>>> streaming infinite time-series (like live video) while ensuring
>> good QoE
>>> despite being low on resource consumption.
>> 
>> But such a multi-feature protocol is also claimed by Google with 
>> QUIC
>> 
>> protocol. Why not QUIC?
>> 
>>> The draft is available here: 
>>> https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-00
>
>>>
>>>
>>>
>>>
>>> 
>> 
>>> Probably, the use case that we had in mind and the technology we 
>>> proposed, both can be extended to the purview of this WG.
>>> 
>>> Requesting your views on this draft please.. If relevant then we
>> can
>>> present this draft in Prague.
>> 
>> I can look at the draft, but I need to understand why CoAP?
>> 
>> Video per se is not a sufficient reason to use CoAP, I think.
>> 
>> (and yes, I do agree there are strong IP requirements for video in 
>> drones, like multi-cameras, multiple laser, infrared, and others)
>> 
>> Alex
>> 
>>> 
>>> Here is the abstract for your ready reference:
>>> 
>>> This draft presents extensions to Constrained Application
>> Protocol
>>> (CoAP) to enable RESTful Real-time Live Streaming for improving
>> the
>>> Quality of Experience (QoE) for delay-sensitive Internet of
>> Things
>>> (IoT) applications. The overall architecture is termed
>> ''Adaptive
>>> RESTful Real-time Live Streaming for Things (A-REaLiST)''. It
>> is
>>> particularly designed for applications which rely on real-time 
>>> augmented vision through live First Person View (FPV) feed from 
>>> constrained remote agents like Unmanned Aerial Vehicle (UAV),
>> etc.
>>> These extensions provide the necessary hooks to help solution 
>>> designers ensure low-latency transfer of streams and, for
>> contents
>>> like video, a quick recovery from freeze and corruption without 
>>> incurring undue lag. A-REaLiST is an attempt to provide an 
>>> integrated approach to maintain the balance amongst QoE,
>> resource-
>>> efficiency and loss resilience. It provides the necessary hooks
>> to
>>> optimize system performance by leveraging contextual
>> intelligence
>>> inferred from instantaneous information segments in flight.
>> These
>>> extensions equip CoAP with a standard for efficient RESTful 
>>> streaming for Internet of Things (IoT) contrary to
>> HTTP-streaming in
>>> conventional Internet.
>>> 
>>> 
>>> 
>>> 
>>> Thank you. Looking forward.
>>> 
>>> P.S. The work has been accepted and presented in Wi-UAV workshop
>>>  in
>> 
>>> Globecom, Dec,  2018. The public link to the paper is yet to be
>> available.
>>> 
>>> With Best Regards Abhijan Bhattacharyya Consultant / Scientist, 
>>> {Internet Protocols | 5G | Standardization}, TCS Research, Tata 
>>> Consultancy Services Building 1B,Ecospace Plot -  IIF/12 ,New 
>>> Town, Rajarhat, Kolkata - 700160,West Bengal India Ph:- +91 33 
>>> 66884691 Cell:- [phone] | [phone] Mailto: 
>>> [e-mail]
> <mailto:[e-mail]>
>> <mailto:[e-mail]>
>>> Website: http://www.tcs.com 
>>> ____________________________________________ Experience 
>>> certainty. IT Services Business Solutions Consulting 
>>> ____________________________________________
>>> 
>>> =====-----=====-----===== Notice: The information contained in 
>>> this e-mail message and/or attachments to it may contain 
>>> confidential or privileged information. If you are not the 
>>> intended recipient, any dissemination, use, review, distribution,
>>> printing or copying of the information contained in this e-mail
>>> message and/or attachments to it are strictly prohibited. If you
>>> have received this communication in error, please notify us by
>>> reply e-mail or telephone and immediately and permanently delete
>>> the message and any attachments. Thank you
>>> 
>>> 
>>> _______________________________________________ its mailing list 
>>> [e-mail] <mailto:[e-mail]> 
>>> https://www.ietf.org/mailman/listinfo/its
>>> 
>> 
>> _______________________________________________ its mailing list 
>> [e-mail] <mailto:[e-mail]> 
>> https://www.ietf.org/mailman/listinfo/its
>> 
>>