Re: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise))

"Timothy B. Terriberry" <tterribe@xiph.org> Mon, 16 March 2015 18:38 UTC

Return-Path: <tterribe@xiph.org>
X-Original-To: video-codec@ietfa.amsl.com
Delivered-To: video-codec@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D98311A8A72 for <video-codec@ietfa.amsl.com>; Mon, 16 Mar 2015 11:38:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.277
X-Spam-Level:
X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qt6xe3uiN5dX for <video-codec@ietfa.amsl.com>; Mon, 16 Mar 2015 11:38:47 -0700 (PDT)
Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA4531A8A6B for <video-codec@ietf.org>; Mon, 16 Mar 2015 11:38:45 -0700 (PDT)
Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 10BDCF217F for <video-codec@ietf.org>; Mon, 16 Mar 2015 11:38:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at mozilla.org
Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RBdf1qEAxE2u for <video-codec@ietf.org>; Mon, 16 Mar 2015 11:38:44 -0700 (PDT)
Received: from [10.252.25.12] (corp.mtv2.mozilla.com [63.245.221.32]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id E6211F214E for <video-codec@ietf.org>; Mon, 16 Mar 2015 11:38:44 -0700 (PDT)
Message-ID: <55072334.1060308@xiph.org>
Date: Mon, 16 Mar 2015 11:38:44 -0700
From: "Timothy B. Terriberry" <tterribe@xiph.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26
MIME-Version: 1.0
To: "video-codec@ietf.org" <video-codec@ietf.org>
References: <CACrD=+_uNyiFuKtoxNYGMOcB2jSvKdh_6mBLvpFeTeEmHDQ9PQ@mail.gmail.com> <CABcZeBOenDx=10XWP_85iSsgGVi0uaTazwfOtN7EsuXS0oUG_A@mail.gmail.com> <CABcZeBPpDNK9+uVmhYo4-LDjkeuNkX86eOsfCzfrO04aeYrYYw@mail.gmail.com> <CACrD=+8wrCc31s1smT5WRkg8t3QO+RnNVSzhwJcvAZzZ6WDFNw@mail.gmail.com> <D0ECB849.42FC4%mzanaty@cisco.com> <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> <CA+E6M0kg782hAhOWx42xFpbgSzQtBHu_JKDjku+CxWvBv4agvA@mail.gmail.com> <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> <550591F2.7020809@xiph.org> <A3F71BD1-78AC-44F7-8C4F-235349DE3706@cisco.com> <5505F464.8080107@alvestrand.no>, <CA+E6M0=HzwVj3_9ShT5KBV0i8+XUBtvGsbj4Hpw==PAz4_hpWw@mail.gmail.com> <73E73F17-6194-426C-B1CE-AE3094EDDF2A@cisco.com>
In-Reply-To: <73E73F17-6194-426C-B1CE-AE3094EDDF2A@cisco.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Archived-At: <http://mailarchive.ietf.org/arch/msg/video-codec/onrQaRYuVAOi2M_TtKjiuLdoWbM>
Subject: Re: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise))
X-BeenThere: video-codec@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Video codec BoF discussion list <video-codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/video-codec>, <mailto:video-codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/video-codec/>
List-Post: <mailto:video-codec@ietf.org>
List-Help: <mailto:video-codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/video-codec>, <mailto:video-codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Mar 2015 18:38:50 -0000

Mo Zanaty (mzanaty) wrote:
> To clarify my all-intra comment, I did indeed mean a test config of all
> intra frames (like jct-vc), but the primary purpose of those results
> would be a separate data point for general intra efficiency in all use

I agree, particularly because there are a number of coding tools which 
one would expect to help primarily on intra frames, and it makes it much 
easier to measure their performance when not cluttered by the noise of 
two orders of magnitude more inter frames (though it's still important 
to test with inter frames to make sure they don't have some deleterious 
effect on prediction).

Personally, I think it's better to have a wide variety of still images 
on which to measure intra performance, rather than coding an existing 
video test sequence with all-intra. The main reason being that 
individual frames in a video clip are highly correlated, but the space 
of images is very large. It's a better use of testing resources to use 
uncorrelated images to explore that space as widely as possible. That's 
what informed the construction of the four still-image subsets in 
draft-daede-netvc-testing Section 5.2.