Re: [Mops] Low latency protocol comparision

[Maxim Sharabayko <[e-mail]>]

Hi,

The idea of performing the comparison is very exciting! I would be glad to participate as a delegate from SRT.

@Glen
It would be great if any SRT folks on this list could do something similar,  to to mention those working with CMAF and LL HLS.

There are of course some inaccuracies in the description of SRT.
The receiver buffer latency of 120 ms is the default value, not the minimum. The minimum is 1 ms.
There is no limit of the number of packet retransmissions, although the configurable limit might be added soon.
HLS and DASH can be transmitted over SRT. And so on.

But in general even with these inaccuracies, the overview of the existing protocols on the market is quite good.

--
Regards,
Maxim Sharabayko
Senior Software Developer | Haivision

From: Mops <[e-mail]> on behalf of Alexandre GOUAILLARD <[e-mail]>
Date: Sunday, 14. June 2020 at 01:28
To: "Deen, Glenn (NBCUniversal)" <[e-mail]>
Cc: Leslie Daigle <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: [Mops] Low latency protocol comparision

@Deen, agreed. I gave it a try though a couple of years back. Let me know if that would go in the right direction and I can update it with everything I learned in the mean time, and taking into account the updates to the different protocols (e.g. new HLS informational draft by pantos):
http://webrtcbydralex.com/index.php/2018/05/15/streaming-protocols-and-ultra-low-latency-including-webrtc/

@Leslie, I volunteer every year, every hackathon :-)
However, I am more knowledgeable about webrtc than other protocols, so in that regard, I am biased.
I think to be fair each protocol should have a reasonable representative/champion that at least validates that the testing protocol chosen is adequate, and garantees the accuracy and the fairness of the result.

For webrtc, it was problematic as well, as each browser vendor did not want someone else to benchmark their implementation, so we ended up with one representant of each browser vendor at 104 I believe (pragues), to validate its result, and did it again in montreal:
https://trac.ietf.org/trac/ietf/meeting/wiki/105hackathon/webrtc
if we manage to get the same setting, I think it could be very successful in getting fair and accurate results.

I think we could handle the webrtc testing (as in writing open source tests and providing results that would be deterministic and independently verifiable by any and all).
- Maybe Veriskope, which is trying to push an RTMP 2.0, could be interested in joining for RTMP.
- Magnus W for RTSP 2.0 ?
- Then we would need one volunteer for SRT,
- and another one for QUIC (maybe martin thompson can point us in the right direction). QUIC is always the biggest table so it should not be too difficult.
- Jonathan Lennox, chairman of AVTCORE is always around and could help, especially on the congestion control, Error concealment, and bandwidth adaptation part
- We could motivate EKR to comment on the security aspects.
- I'm not sure if we want to asses IPv6 and NAT traversal readiness (ICE)

What do you think?

Regards,

Dr. Alex






On Sun, Jun 14, 2020 at 1:05 AM Deen, Glenn (NBCUniversal) <[e-mail]<mailto:[e-mail]>> wrote:
Yes, it’s a bit marketing, but it’s also one of the few write ups that attempts to put the popular media transports side by side for people to understand their strengths and abilities.   The marketing blogs also give us, the IETF, insight into how the outside world of adopters and sellers of our work see things.

I really wish that all web documents and pages contained a posting date.   This one doesn’t gave it a date.  However, the page must be from at least mid 2019 as it mentions things that were released then.

I like your write up of the WebRTC inaccuracies.  It would be great if any SRT folks on this list could do something similar,  to to mention those working with CMAF and LL HLS.  There a lot of adopters out there who are working through what protocol best fits their particular needs - webRTC?   SRT?  Something that bright summer intern wrote while she was here last summer?

 Capturing something like this, and even better some sort of bake off as you suggest would be valuable to a lot of people.  Though it may have to wait to get critical mass of participants until next year.

Glenn

On Jun 13, 2020, at 3:34 PM, Alexandre GOUAILLARD <[e-mail]<mailto:[e-mail]>> wrote:
There are multiple inaccurate statements in the linked documents. Most of them that have been carried around for many years by other SRT vendors like Wowza and already called out (here<http://webrtcbydralex.com/index.php/2019/05/18/wowzas-marketing-at-work-again-webrtc/> for example).

I'm just going to point to three of the most egregious (read "verifiably wrong") statements, that by themselves undermine the credibility of the entire document.

- "Maximum supported resolution: 720p, 30 frames per second with a bit rate of up to 2 Mbps."
- There is no limit to the spatial and temporal resolution in webrtc/rtcweb protocol.
Stadia for example send up to 4K@30fps.

"WebRTC is inferior to its colleagues in terms of the coding quality and maximum amount of transmitted data.
- There is no limit to the bandwidth used in the webrtc/rtcweb protocol.
- There is a limit of 2Mbps per peer connection on the sending-side only in the google chrome implementation, only by default (web application can lift this limit), and only for webcams devices (screensharing has no bandwidth limit). This is a chrome implementation detail, and not a protocol limit.
-While VP8 and H264 are both mandatory to implement (original IETF discussion<https://mailarchive.ietf.org/arch/msg/rtcweb/juVsZKp-AmRpYLCgy-uwartuNrY/>) anybody can add any codec to webrtc as long as an RTP payload specification exists.
-- Google chrome, and firefox, both supports real-time VP9 mode 2 (4:4:4, 10 bits, HDR)
https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/video_coding/codecs/vp9/vp9_impl.cc;l=985
-- Chrome has a real-time AV1 implementation in webrtc, which goes up to high profile (4:4:4, 8~10bits, HDR).
https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/video_coding/codecs/av1/
In terms of quality, only codecs at high profile (i.e. 12bits) can do better, and none of those have a "real-time" version today.
In terms of coding efficiency, AV1 is still the best codec available in production.

"WebRTC is not available in Safari and partially unavailable in Bowser and Edge."
- webrtc is fully available in Edge, and Safari.
https://caniuse.com/#search=webrtc
official daily W3C results: https://wpt.fyi/results/?label=master&label=experimental&aligned&q=webrtc
- Safari has added H.265 support in addition to the mandatory to implement VP8 and H..264 in the latest safari tech preview.

I'm not going into the Discussion about webrtc's underlying RTC/RTCP protocol and corresponding extension: NACK, RTX, FEC, RED, or Congestion Control (REMB, Google-cc, BBR, NADA), as IETF members I'm sure you re all aware of RFC3550 and all its derivatives. Many authors of those RFCs are part of this group.

Maybe more overlooked is the most recent Draft for end-to-end encryption on top of webrtc co-authored by google:
https://tools.ietf.org/html/draft-omara-sframe-00

I'm not saying that SRT is worse, or better than WebRTC.. I'm just saying, this comparison is not egregiously incorrect, and I'm a little bit surprised that we are discussing marketing blogs that go in the face of so many IETF specs.

If anybody is interested in going deep into a real  thorough comparison between all those protocols, why not doing it during the IETF hackathon in July, or in october (hopefully in BKK) just like we did group co-testing with IETF's AVTCORE/AVEXT/QUIC/RTCWEB/RMCAT during the previous IETF hackathons?

Regards,



On Sat, Jun 13, 2020 at 6:58 PM Leslie Daigle <[e-mail]<mailto:[e-mail]>> wrote:

I thought this was a pretty interesting analysis:

Low broadcast latency and protocols for implementation thereof:
https://www.elecard.com/page/low_broadcast_latency_and_protocols_for_implementation_thereof

Particularly telling is the chart at the end: WebRTC and SRT don’t look very similar, in this particular regard.

Leslie.

--

________________________________

Leslie Daigle
Principal, ThinkingCat Enterprises

[e-mail]<mailto:[e-mail]>
--
Mops mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/mops


--
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard<http://sg.linkedin.com/agouaillard>
·
--
Mops mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/mops


--
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard<http://sg.linkedin.com/agouaillard>
·