Re: [Sframe] Partial decodability and IDUs

Justin Uberti <> Thu, 19 November 2020 21:54 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 9C97A3A12B4 for <>; Thu, 19 Nov 2020 13:54:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -17.599
X-Spam-Status: No, score=-17.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id HXkIsYWN7tyy for <>; Thu, 19 Nov 2020 13:54:13 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 0FEAA3A12B2 for <>; Thu, 19 Nov 2020 13:54:13 -0800 (PST)
Received: by with SMTP id y18so6717716ilp.13 for <>; Thu, 19 Nov 2020 13:54:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CWoLRhMx5J6RkzmPehLs20k14J18ngoZZ0g9R4fBSWA=; b=t7/8cO3TCk4aSOXUA10IlIYocJSgAsfU9fL1aq5JEGNb64QShCYRkyA4fzTIYWhtE5 Xbd4/DX3WKh9P99s9abMtV0ugMXxI2M4ZuieUA/afHekI5RGq68a98mjYQdpdzPUHg1v L73zwlDfzcH01eIjC8evhBm09NwGDepUiMYmT1ZlwIH0sVeLVeY5s9wDHYMxfKukvbjf Ko15vsf8WidyiReWotj1CbISvH8cVurtTzhPusindJHM81OJqz541Q7QDwhsEnHIu9Tw j59UngxTlJSTs7z3GjuRHWfrzCZLZ+zXaS63lTlOOTn+bXDL/9yqlgUA7vJIv6bdULvs M6Bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CWoLRhMx5J6RkzmPehLs20k14J18ngoZZ0g9R4fBSWA=; b=MUI2xhRY5Z8NaY8HSls8WpSduE+L7otwPla//QWttfkTcidi7jQvNWKpnLpwp62Trl /Zic10aR3EIo8/RlJv7/Zc8FTzOnxOUY8hJ1YHAhCQpkZwlposKPZp33yp5Olylkc08t MSa5oRbDS3AV0PxVqIlzz1Ztio+mAXyJAuohOu1v5aHjCOr4hGU2wiJN6H+U0G/oSXsN B6P3SYYW/6jDAZj0EnUc9zykdgtmyHiHQyLPi0WtqQ4CIAK0XsyNm2qXMxWdkA92PJHi OXp/pNymupg3seeTCikXEVGRrES7jBMxa5q4I67JW/AeQ8XsFV/gqwRAykBBQZAuSbCa DVVw==
X-Gm-Message-State: AOAM530kzj/PFSLOSkBIaMyFsb0lwfMw3L5NvN2pPR8IcOgsPlhcre2W k4yIKcioqtzvS4P5WiKqtLML0ETg6Z9V1TfVE0S6zw==
X-Google-Smtp-Source: ABdhPJz6fMallAmuk5pnGoQv6uvOAMFGUmuFPM/rfY3Loclt2nTdt0ybAKs8CXOBHWTnOJp6h+ZjHPgRqD1V+aAUAvg=
X-Received: by 2002:a92:db0b:: with SMTP id b11mr14196161iln.55.1605822851977; Thu, 19 Nov 2020 13:54:11 -0800 (PST)
MIME-Version: 1.0
References: <>
In-Reply-To: <>
From: Justin Uberti <>
Date: Thu, 19 Nov 2020 13:53:57 -0800
Message-ID: <>
To: Sergio Garcia Murillo <>
Cc: "" <>
Content-Type: multipart/alternative; boundary="000000000000aa796c05b47cc4dc"
Archived-At: <>
Subject: Re: [Sframe] Partial decodability and IDUs
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 19 Nov 2020 21:54:15 -0000

The encoder needs to be aware of any mechanism to generate IDUs (e.g.,
slices), and typically each of these IDUs will be handed up to the consumer
individually. So the SFRAME layer doesn't need to do any splitting, it just
knows that it should treat each IDU as something it needs to individually
SFRAME and packetize.

On Thu, Nov 19, 2020 at 1:40 PM Sergio Garcia Murillo <> wrote:

> Hi all,
> As most of you already know, this morning I made a presentation in AVTCORE
> introducing the topic about the need to specify an agnostic video codec
> packetization format.
> I got an AP for creating an initial draft so it could be reviewed and
> accepted.
> However, there were two main concerns that we should address in this this
> group:
>    - Historically, avtcore has explicitly designed not to be payload
>    agnostic and  declined to standardized codec agnostic payload formats in
>    number of cases.  If that is to be changed, needs to be done deliberately.
>    - Need to define the "minimum decoding unit" or "independently
>    decodable unit", that SFrame will work with.
> Regarding the second one
>    - Full video frames (just use whatever is the encoder output)
>    - Spatial layer frames
>    - "independend decodable subframes" like h264 slices, vp8 partitions
>    or av1 tiles which allows partial decodability which is mainly aimed for
>    enhancing packet loss resilience.
> Spatial layer frames is the minimum we should target as if not it will
> just prevent SFUs for using SVC codecs. So the question is if we should go
> deeper and implement lower partitions of the frames or not.
> AFAIK, currently, libwertc does not support partial decodability and I
> personally haven't seen any practical usage of this in the RTC world (while
> it makes a lot of sense in streaming/broadcasting world), but would like to
> hear what is the view and experience of the other members of this group.
> Also note that if we are going to support them on SFrame this will require
> a greater effort because we will need to explicitly define how the frames
> must be split before being encrypted y SFrame for *each* possible video
> codec (h264,h265,vp8,vp9,av1,...).
> There was also the question about how/if we should support other codec
> features like DON/interleaved mode for h264, which I also think we should
> not support mainly because we are not currently using it on webrtc
> implementations.
> What do you think?
> Best regards
> Sergio
> --
> Sframe mailing list