[Mcast-wifi] Video quality with GroupCast with Retries (GCR): some results

PABLO SERRANO YAÑEZ-MINGOT <pablo@it.uc3m.es> Thu, 15 November 2018 07:12 UTC

Return-Path: <pablo@it.uc3m.es>
X-Original-To: mcast-wifi@ietfa.amsl.com
Delivered-To: mcast-wifi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E36E130EDA for <mcast-wifi@ietfa.amsl.com>; Wed, 14 Nov 2018 23:12:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level:
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=it-uc3m-es.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZslPWYVpX-HK for <mcast-wifi@ietfa.amsl.com>; Wed, 14 Nov 2018 23:12:07 -0800 (PST)
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com [IPv6:2a00:1450:4864:20::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D114D130DC6 for <Mcast-wifi@ietf.org>; Wed, 14 Nov 2018 23:12:03 -0800 (PST)
Received: by mail-lj1-x229.google.com with SMTP id v15-v6so16360945ljh.13 for <Mcast-wifi@ietf.org>; Wed, 14 Nov 2018 23:12:03 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=it-uc3m-es.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=3UyqPtQil4wIjglrzJUOVeaLz/KlfGhH6plAoUhre/k=; b=xcRSAY7hbvPBqiFZIi2ejLtsmD/LhUrRJW0iPBLmhWsLW43FkCGCRG0/u0v9hRjNLg wIHC4eU1wIzCdSTQoXs3ihsmUzryRredND3K7IkNPdqrjzvxIWmpy1w7XzjuCt6vtIoT 9ITdW5+ydU69iwpGmcYm6GoQMRt0A+kE/c+w+tNyEL50o4iS7//P1IF+ZrJOz95hdRGa UYry+wuTGJ9aJ70QHpBUEizbr4BNDvDFM3T7zohvbgBQnQ29MX2PoV07Ag4ZXUhyzOgh TfG0W+kQ745QbM7rOFSZCC/f2VbfxmSQTheOVEhOzaleVjodKIE9Bc6hNa7LZqph/hzh Vp9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=3UyqPtQil4wIjglrzJUOVeaLz/KlfGhH6plAoUhre/k=; b=acStkpY9GqkIEeGcuSId8zgSnVhHQrhJuj6epUZ7d1bocik8XLwc7w1vb8SGHwOKvm t3lF9+IAtKDedM5xGRQbAtKQ1LNgmMAZa5+HSv1fzuSXFdWD4X6ygt05dvDNTqzRcdhj JfvAMGFYttablcWPOtL2AQ1gY33Wb9v8We87E+QTp7kZ8313IDdDya0bI+F+Kyld/FfR HE8XQya7rCjN5xN053Iikl8ZTBWf07Gi7jxi0e1lilMutwKgBvFFyBB5/qObFTrrkphj A/eeqQNeN64A8L7XTrlosN1epsC0JNFv8s7GqFVfcub7adU1qEz66B5SOcoN7Ej61HLg GKyg==
X-Gm-Message-State: AGRZ1gJ7AwbSuqj1+X4WwEE/KMWUF7ygMpR+Tm1ALljkB/X0TGxTYLsw pBptRFe+70ju/fL3AtUWJR+f2/byo19NT3Szjxd9eUQQ8eo=
X-Google-Smtp-Source: AJdET5e7tiBulK3V9JtBq7wdhWsC016eyXrDnR36lv7nH6dfaIBbQFo5P9TkJPJSR6b8PloLKhwwoZopfA19P0hdhG0=
X-Received: by 2002:a2e:3218:: with SMTP id y24-v6mr3227135ljy.157.1542265921437; Wed, 14 Nov 2018 23:12:01 -0800 (PST)
MIME-Version: 1.0
From: =?UTF-8?Q?PABLO_SERRANO_YA=C3=91EZ=2DMINGOT?= <pablo@it.uc3m.es>
Date: Thu, 15 Nov 2018 08:11:49 +0100
Message-ID: <CAMnyNjMRGCMe63_FgzBGg0zakp20E0MSXDCCZbFgQewKXF1ZuA@mail.gmail.com>
To: Mcast-wifi@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/mcast-wifi/kEMK-BLex4Q9REWwzRs981nxxVM>
Subject: [Mcast-wifi] Video quality with GroupCast with Retries (GCR): some results
X-BeenThere: mcast-wifi@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions related to issues with multicast in 802.11 Wi-Fi networks & solutions/optimizations targeted at resolving these issues." <mcast-wifi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mcast-wifi>, <mailto:mcast-wifi-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mcast-wifi/>
List-Post: <mailto:mcast-wifi@ietf.org>
List-Help: <mailto:mcast-wifi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mcast-wifi>, <mailto:mcast-wifi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Nov 2018 07:12:14 -0000

Hi all,

We've been experimenting for some time with the delivery of multicast
video, comparing the performance of the legacy 802.11 behaviour and
the novel GCR mechanisms, namely, UR and BA (in fact, the draft
[Perkins2018] cites a work we did some years ago, Oliva2013)

Instead of objective metrics such as jitter or packet loss, we've
estimated the "Quality of Experience" (QoE) using Netflix's VMAF tool
[VMAF]. While the scope of the study is of course limited to our
scenario (academic environment, 802.11g, 25 receivers), some take-away
results that could be of interest to the draft are (full paper
[Gringoli2018]):

* A loss of 5% (as mentioned in Section 3.1.1 of the draft) is way too
high for a watchable video experience. According to our results, even
with losses around 1% or less the QoE could be pretty bad.

* Unsolicited Retries (mentioned in Section 4.7) does improve a bit
performance, but fails to provide an adequate QoE. For video delivery
to multiple receivers, better to stick to BA.

* The use of the default EDCA differentiation slightly improves
performance, but if the video quality was already bad, it won't result
in a watchable experience.

Rgds, Pbl

References:

[Perkins2018] https://datatracker.ietf.org/doc/draft-ietf-mboned-ieee802-mcast-problems/

[Gringoli2018] http://doi.org/10.1109/TMC.2018.2876000

[VMAF] https://github.com/Netflix/vmaf