Re: [Mlcodec] Opus extensions discussion points

Jean-Marc Valin <jmvalin@amazon.com> Fri, 20 October 2023 18:03 UTC

Return-Path: <prvs=650e684e9=jmvalin@amazon.com>
X-Original-To: mlcodec@ietfa.amsl.com
Delivered-To: mlcodec@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8EB94C14CEFE for <mlcodec@ietfa.amsl.com>; Fri, 20 Oct 2023 11:03:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.402
X-Spam-Level:
X-Spam-Status: No, score=-4.402 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazon.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZsWVLlroYTqw for <mlcodec@ietfa.amsl.com>; Fri, 20 Oct 2023 11:03:41 -0700 (PDT)
Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C2F63C14CF17 for <mlcodec@ietf.org>; Fri, 20 Oct 2023 11:03:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1697825021; x=1729361021; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=FYbIVap8EoPjtZn3UM1FEm77KfBt8n5pPjOv6vWKBNY=; b=m4Gk/1fNP50jvzxyUII51Fyuz1/Z8HosV35+kpXtn7t+/bNUKNbS21Fz XFOCEBcKZZrGR+mdSKLq2YFuaJx7+Iupeu1bu4OB6SDIUj2LxfXEdufal K68yRQQfk8Jxx2B+Eb0w3rCRzV7C1uIqlQCxXhLDqloFu0UD1WO4z0Z1x A=;
X-IronPort-AV: E=Sophos;i="6.03,239,1694736000"; d="scan'208";a="357950065"
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-1197e3af.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-2101.iad2.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 18:03:40 +0000
Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2a-m6i4x-1197e3af.us-west-2.amazon.com (Postfix) with ESMTPS id BDF6B100447; Fri, 20 Oct 2023 18:03:38 +0000 (UTC)
Received: from EX19MTAUEA002.ant.amazon.com [10.0.29.78:22753] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.45.91:2525] with esmtp (Farcaster) id db782dc9-98b6-4f1e-b944-8111ea367b54; Fri, 20 Oct 2023 18:03:38 +0000 (UTC)
X-Farcaster-Flow-ID: db782dc9-98b6-4f1e-b944-8111ea367b54
Received: from EX19D015UEA003.ant.amazon.com (10.252.134.165) by EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.37; Fri, 20 Oct 2023 18:03:37 +0000
Received: from [192.168.13.122] (10.187.170.20) by EX19D015UEA003.ant.amazon.com (10.252.134.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Fri, 20 Oct 2023 18:03:36 +0000
Message-ID: <4a685159-2fd2-45e0-a90d-705b95de4618@amazon.com>
Date: Fri, 20 Oct 2023 14:03:32 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: Jonathan Lennox <jonathan.lennox=408x8.com@dmarc.ietf.org>, "Timothy B. Terriberry" <tterribe@xiph.org>
CC: mlcodec@ietf.org
References: <f76b0356-fde4-64ec-655d-5d45cb780e20@xiph.org> <3B470846-9638-4CE3-B02A-55DB03044D6A@8x8.com>
From: Jean-Marc Valin <jmvalin@amazon.com>
In-Reply-To: <3B470846-9638-4CE3-B02A-55DB03044D6A@8x8.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: base64
X-Originating-IP: [10.187.170.20]
X-ClientProxiedBy: EX19D037UWC001.ant.amazon.com (10.13.139.197) To EX19D015UEA003.ant.amazon.com (10.252.134.165)
Archived-At: <https://mailarchive.ietf.org/arch/msg/mlcodec/8UbgjqOXJ04JSG2wWdldWSFWnnY>
Subject: Re: [Mlcodec] Opus extensions discussion points
X-BeenThere: mlcodec@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Machine Learning for Audio Coding <mlcodec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mlcodec>, <mailto:mlcodec-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mlcodec/>
List-Post: <mailto:mlcodec@ietf.org>
List-Help: <mailto:mlcodec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mlcodec>, <mailto:mlcodec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 20 Oct 2023 18:03:41 -0000

The idea -- assuming there's a use for it -- would be to say "if you 
don't understand this extension, discard all extensions" but the 
baseline Opus would still be decodable. A theoretical use case would be 
an extension that signals "do the opposite of what the other extensions 
say" or "the other extensions all use a different format". That's in 
theory anyway. In practice I have a hard time thinking of a valid use 
case, but if anyone can think of one then it's a mechanism we define. If 
not, then no need to bother with unsafe extensions.

     Jean-Marc


On 10/20/23 13:17, Jonathan Lennox wrote:
>> On Oct 19, 2023, at 8:24 PM, Timothy B. Terriberry <tterribe@xiph.org> wrote:
>>
>> = Should we predefine "unsafe" extensions? =
>>
>> I'm the one who raised this question after reading up on another current effort to add extensions to a rather old protocol [2], in an attempt to see if any institutional wisdom from other groups could inform our own efforts. These are extensions that "are not safe to ignore", and UDP is reserving a rather large portion of its extension ID space for them (although the only one currently defined is for encryption). In UDP they are only allowed to be used with fragmentation, so the packet they are in would have been silently dropped if fragmentation was not supported anyway. We don't really have an equivalent in Opus. One could maybe consider something like stereo data or higher audio bandwidths as being information that will be silently dropped if a decoder does not support them, but for an extension that affects just these, I would think that you could encode the regular Opus packet in mono or at the lower audio bandwidth and then the extension becomes safe to ignore. If someone else can see use cases here and thinks we should reserve space for these, please speak up.
> I’m not sure what the point of such extensions would be.  If extensions are unsafe to ignore, you have to ensure all receivers understand it, at which point you’re defining a new codec.  Negotiating such things seems like it’d be much simpler to do as simply a new codec media type, which would probably also allow you to be more bit-efficient with it as well.
>
> --
> Mlcodec mailing list
> Mlcodec@ietf.org
> https://www.ietf.org/mailman/listinfo/mlcodec