Re: [Mops] Low latency protocol comparision

[Alexandre GOUAILLARD <[e-mail]>]

@Deen, agreed. I gave it a try though a couple of years back. Let me know
if that would go in the right direction and I can update it with everything
I learned in the mean time, and taking into account the updates to the
different protocols (e.g. new HLS informational draft by pantos):
http://webrtcbydralex.com/index.php/2018/05/15/streaming-protocols-and-ultra-low-latency-including-webrtc/

@Leslie, I volunteer every year, every hackathon :-)
However, I am more knowledgeable about webrtc than other protocols, so in
that regard, I am biased.
I think to be fair each protocol should have a
reasonable representative/champion that at least validates that the testing
protocol chosen is adequate, and garantees the accuracy and the fairness of
the result.

For webrtc, it was problematic as well, as each browser vendor did not want
someone else to benchmark their implementation, so we ended up with one
representant of each browser vendor at 104 I believe (pragues), to validate
its result, and did it again in montreal:
https://trac.ietf.org/trac/ietf/meeting/wiki/105hackathon/webrtc
if we manage to get the same setting, I think it could be very successful
in getting fair and accurate results.

I think we could handle the webrtc testing (as in writing open source tests
and providing results that would be deterministic and
independently verifiable by any and all).
- Maybe Veriskope, which is trying to push an RTMP 2.0, could be interested
in joining for RTMP.
- Magnus W for RTSP 2.0 ?
- Then we would need one volunteer for SRT,
- and another one for QUIC (maybe martin thompson can point us in the right
direction). QUIC is always the biggest table so it should not be too
difficult.
- Jonathan Lennox, chairman of AVTCORE is always around and could help,
especially on the congestion control, Error concealment, and bandwidth
adaptation part
- We could motivate EKR to comment on the security aspects.
- I'm not sure if we want to asses IPv6 and NAT traversal readiness (ICE)

What do you think?

Regards,

Dr. Alex






On Sun, Jun 14, 2020 at 1:05 AM Deen, Glenn (NBCUniversal) <
[e-mail]> wrote:

> Yes, it’s a bit marketing, but it’s also one of the few write ups that
> attempts to put the popular media transports side by side for people to
> understand their strengths and abilities.   The marketing blogs also give
> us, the IETF, insight into how the outside world of adopters and sellers of
> our work see things.
>
> I really wish that all web documents and pages contained a posting date.
> This one doesn’t gave it a date.  However, the page must be from at least
> mid 2019 as it mentions things that were released then.
>
> I like your write up of the WebRTC inaccuracies.  It would be great if any
> SRT folks on this list could do something similar,  to to mention those
> working with CMAF and LL HLS.  There a lot of adopters out there who are
> working through what protocol best fits their particular needs - webRTC?
> SRT?  Something that bright summer intern wrote while she was here last
> summer?
>
>  Capturing something like this, and even better some sort of bake off as
> you suggest would be valuable to a lot of people.  Though it may have to
> wait to get critical mass of participants until next year.
>
> Glenn
>
> On Jun 13, 2020, at 3:34 PM, Alexandre GOUAILLARD <[e-mail]>
> wrote:
>
> 
> There are multiple inaccurate statements in the linked documents. Most of
> them that have been carried around for many years by other SRT vendors like
> Wowza and already called out (here
> <http://webrtcbydralex.com/index.php/2019/05/18/wowzas-marketing-at-work-again-webrtc/>
> for example).
>
> I'm just going to point to three of the most egregious (read "verifiably
> wrong") statements, that by themselves undermine the credibility of the
> entire document.
>
> - "Maximum supported resolution: 720p, 30 frames per second with a bit
> rate of up to 2 Mbps."
> - There is no limit to the spatial and temporal resolution in
> webrtc/rtcweb protocol.
> Stadia for example send up to 4K@30fps.
>
> "WebRTC is inferior to its colleagues in terms of the coding quality and
> maximum amount of transmitted data.
> - There is no limit to the bandwidth used in the webrtc/rtcweb protocol.
> - There is a limit of 2Mbps per peer connection on the sending-side *only
> in the google chrome implementation*, only by default (web application
> can lift this limit), and only for webcams devices (screensharing has no
> bandwidth limit). This is a chrome implementation detail, and not a
> protocol limit.
> -While VP8 and H264 are both mandatory to implement (original IETF
> discussion
> <https://mailarchive.ietf.org/arch/msg/rtcweb/juVsZKp-AmRpYLCgy-uwartuNrY/>)
> anybody can add any codec to webrtc as long as an RTP payload specification
> exists.
> -- Google chrome, and firefox, both supports real-time VP9 mode 2 (4:4:4,
> 10 bits, HDR)
>
> https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/video_coding/codecs/vp9/vp9_impl.cc;l=985
> -- Chrome has a real-time AV1 implementation in webrtc, which goes up to
> high profile (4:4:4, 8~10bits, HDR).
>
> https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/video_coding/codecs/av1/
> In terms of quality, only codecs at high profile (i.e. 12bits) can do
> better, and none of those have a "real-time" version today.
> In terms of coding efficiency, AV1 is still the best codec available in
> production.
>
> "WebRTC is not available in Safari and partially unavailable in Bowser and
> Edge."
> - webrtc is fully available in Edge, and Safari.
> https://caniuse.com/#search=webrtc
> official daily W3C results:
> https://wpt.fyi/results/?label=master&label=experimental&aligned&q=webrtc
> - Safari has added H.265 support in addition to the mandatory to implement
> VP8 and H..264 in the latest safari tech preview.
>
> I'm not going into the Discussion about webrtc's underlying RTC/RTCP
> protocol and corresponding extension: NACK, RTX, FEC, RED, or Congestion
> Control (REMB, Google-cc, BBR, NADA), as IETF members I'm sure you re all
> aware of RFC3550 and all its derivatives. Many authors of those RFCs are
> part of this group.
>
> Maybe more overlooked is the most recent Draft for end-to-end encryption
> on top of webrtc co-authored by google:
> https://tools.ietf.org/html/draft-omara-sframe-00
>
> I'm not saying that SRT is worse, or better than WebRTC.. I'm just saying,
> this comparison is not egregiously incorrect, and I'm a little bit
> surprised that we are discussing marketing blogs that go in the face of so
> many IETF specs.
>
> If anybody is interested in going deep into a real  thorough comparison
> between all those protocols, why not doing it during the IETF hackathon in
> July, or in october (hopefully in BKK) just like we did group co-testing
> with IETF's AVTCORE/AVEXT/QUIC/RTCWEB/RMCAT during the previous IETF
> hackathons?
>
> Regards,
>
>
>
> On Sat, Jun 13, 2020 at 6:58 PM Leslie Daigle <[e-mail]>
> wrote:
>
>> I thought this was a pretty interesting analysis:
>>
>> Low broadcast latency and protocols for implementation thereof:
>>
>> https://www.elecard.com/page/low_broadcast_latency_and_protocols_for_implementation_thereof
>>
>> Particularly telling is the chart at the end: WebRTC and SRT don’t look
>> very similar, in this particular regard.
>>
>> Leslie.
>>
>> --
>> ------------------------------
>>
>> Leslie Daigle
>> Principal, ThinkingCat Enterprises
>> [e-mail]
>> --
>> Mops mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/mops
>>
>
>
> --
> Alex. Gouaillard, PhD, PhD, MBA
>
> ------------------------------------------------------------------------------------
> President - CoSMo Software Consulting, Singapore
>
> ------------------------------------------------------------------------------------
> sg.linkedin.com/agouaillard
>
>    -
>
> --
> Mops mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/mops
>
>

-- 
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard

   -