[video-codec] Command-line options for encoder comparisons

Thomas Daede <tdaede@mozilla.com> Wed, 25 March 2015 23:19 UTC

Return-Path: <tdaede@mozilla.com>
X-Original-To: video-codec@ietfa.amsl.com
Delivered-To: video-codec@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com []) by ietfa.amsl.com (Postfix) with ESMTP id A3FE61B2A07 for <video-codec@ietfa.amsl.com>; Wed, 25 Mar 2015 16:19:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 3XYG3eF-zIfd for <video-codec@ietfa.amsl.com>; Wed, 25 Mar 2015 16:19:51 -0700 (PDT)
Received: from mail-pd0-f179.google.com (mail-pd0-f179.google.com []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F13A1A90E8 for <video-codec@ietf.org>; Wed, 25 Mar 2015 16:19:51 -0700 (PDT)
Received: by pdbop1 with SMTP id op1so43257604pdb.2 for <video-codec@ietf.org>; Wed, 25 Mar 2015 16:19:51 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=0VM52ZvtOmOXin6hz1nbrf4B7a76JhnVaKAx9HXh1xc=; b=aMlx+n2gvp2sMsToy1U2sNCBMYK7Xeqh8KosQnlvvLRR5w8h7yITJq6anxoTKTT6J1 /nP9nq+05I7Wdop4SBqsA9QlyPMQtrAfec0P/Wo+/P/T2BEwne2FPm+SZhXOhZA9kzQq uD8JzFFueWLDWikHiNXQFs1bgrOFaMcZck6RKG/inDUdNrETuwIjXVPUS8gqAyH+FpmF bp76NRqHWdphZOXBLrwfGMfDsrBbVgor9mA2cBnrKv+zDR1nWMKq1FIxdIbOraCuFMBH JU2S/cTHMgnMDER785DBIO1kcEcsd9Z03mmR6n/9xmLE4S9weblXLjecLDd0wm7Ajm0h wZvw==
X-Gm-Message-State: ALoCoQla9XX9lygm1RVMTESADA7igh0y2c64uWs1twPbNwAMlSktd/L5iXEBSvEszVS/DEGPyleB
X-Received: by with SMTP id 16mr21751991pdf.78.1427325591220; Wed, 25 Mar 2015 16:19:51 -0700 (PDT)
Received: from ?IPv6:2620:101:80fc:224:7e7a:91ff:fe9e:8126? ([2620:101:80fc:224:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id ox4sm3541570pdb.36.2015. for <video-codec@ietf.org> (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Mar 2015 16:19:50 -0700 (PDT)
Message-ID: <55134294.2030803@mozilla.com>
Date: Wed, 25 Mar 2015 16:19:48 -0700
From: Thomas Daede <tdaede@mozilla.com>
User-Agent: http://a.pomf.se/cszdno.opus
MIME-Version: 1.0
To: video-codec@ietf.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
Archived-At: <http://mailarchive.ietf.org/arch/msg/video-codec/WB_MEczty_YtqvwOFiiDXEX4G10>
Subject: [video-codec] Command-line options for encoder comparisons
X-BeenThere: video-codec@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Video codec BoF discussion list <video-codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/video-codec>, <mailto:video-codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/video-codec/>
List-Post: <mailto:video-codec@ietf.org>
List-Help: <mailto:video-codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/video-codec>, <mailto:video-codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Mar 2015 23:19:54 -0000

I am updating the scripts that AreWeCompressedYet uses. I would like to
select what rate control options are used in the comparison. This is for
purposes of absolute comparison between different codecs, and to
indicate when a particular performance point is reached. This is not for
adding tools to an existing codec. There are a couple of different rate
control modes that we can use to compare:

Constant Quantizer:
This is not as simple as it might sound. Should referenced frames still
be allowed a boost in this mode? Should adaptive quantization be
allowed? Can quantization matrices be switched? I tried to come up with
a sane way to compare different codecs with a constant quantizer and was
not successful.

Constant Quality:
Here the codecs are set to use all features available within temporal
constraints. For stored video, this means full lookahead (or two pass,
if that's what the codec uses). For realtime, this means no lookahead,
but otherwise all quality-oriented rate control is allowed. Example rate
control parameters:

VP9 parameters:
vpxenc --codec=vp9 --cq-level=$x --end-usage=q

x264 parameters:
x264 --crf=$x

x265 parameters:
x265 --crf=$x

Constant Bitrate:
This is the hardest category to compare, as it means that all of the CBR
implementations need to be set to have the same buffer constraints. It
is an important use case, so I think we will need to figure out a way to
test it. I'd be okay with adding this one later, though.