[rtcweb] OS Codec APIs

"Mo Zanaty (mzanaty)" <mzanaty@cisco.com> Tue, 05 November 2013 09:52 UTC

Return-Path: <mzanaty@cisco.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 8D97A11E81A9 for <rtcweb@ietfa.amsl.com>; Tue, 5 Nov 2013 01:52:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.598
X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id jD4ZqbtnaazK for <rtcweb@ietfa.amsl.com>; Tue, 5 Nov 2013 01:52:09 -0800 (PST)
Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com []) by ietfa.amsl.com (Postfix) with ESMTP id ED0AB21E80BB for <rtcweb@ietf.org>; Tue, 5 Nov 2013 01:52:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=9398; q=dns/txt; s=iport; t=1383645129; x=1384854729; h=from:to:subject:date:message-id:in-reply-to:mime-version; bh=Wwl8V37lYiOYmorDuJYZ57VBjmIfFzZJNlgmGYthhmQ=; b=TDlTyVb4OnrOA8P0iogUlHDwTm8s5XRUC9ulJ+lbnhWde51yiT7sVCJI b9n113lmW0mAQIlgcEclnJT6tqYTZSE5/Qk8WUgwR8cXlDEZS18q/hR0k VXnuFf9wtjafu3C36TOzyIuNWvJ0bHPo3C+OVFFwmQnp2S9nuPWAMVaHE w=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos; i="4.93,638,1378857600"; d="scan'208,217"; a="280820608"
Received: from rcdn-core2-4.cisco.com ([]) by rcdn-iport-8.cisco.com with ESMTP; 05 Nov 2013 09:52:08 +0000
Received: from xhc-rcd-x13.cisco.com (xhc-rcd-x13.cisco.com []) by rcdn-core2-4.cisco.com (8.14.5/8.14.5) with ESMTP id rA59q73u013663 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 5 Nov 2013 09:52:07 GMT
Received: from xmb-rcd-x14.cisco.com ([]) by xhc-rcd-x13.cisco.com ([]) with mapi id 14.03.0123.003; Tue, 5 Nov 2013 03:52:07 -0600
From: "Mo Zanaty (mzanaty)" <mzanaty@cisco.com>
To: Randell Jesup <randell-ietf@jesup.org>, "rtcweb@ietf.org" <rtcweb@ietf.org>
Thread-Topic: OS Codec APIs
Thread-Index: AQHO2gyw8j4+0037VkqJng1+MvnW/w==
Date: Tue, 5 Nov 2013 09:52:06 +0000
Message-ID: <CE9DD01F.1B8E7%mzanaty@cisco.com>
In-Reply-To: <5277262E.3050600@jesup.org>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_CE9DD01F1B8E7mzanatyciscocom_"
MIME-Version: 1.0
Subject: [rtcweb] OS Codec APIs
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2013 09:52:14 -0000

Khronos OpenMAX IL is the de facto standard for how all chipset vendors expose their codecs to the OS/platform. The problem is the OS/platform does not usually expose those same codecs up to apps with the same API richness (or at all). There are unofficial/hacky ways to access this in Android via IOMX (NDK), or iOS via AV Foundation, as well as official ways in Android 4.1+ via MediaCodec (SDK). But they typically lack the richness of the underlying OMX IL from the chipset vendors, especially real-time and error resilience features.

In my experience with the dominant codec silicon vendors (QCOM, IMG, TI, etc.), their OMX IL implementations are quite robust but not fully interchangeable, i.e. the OS/platform (or app) must cope with different feature sets across the various vendor components.


On 11/3/13, 11:44 PM, Randell Jesup <randell-ietf@jesup.org<mailto:randell-ietf@jesup.org>> wrote:

On 11/3/2013 12:34 PM, DRAGE, Keith (Keith) wrote:
The only stuff I am aware of is done by


But that is as far as my knowledge goes.

My understanding from someone at Skype who worked with OMX stuff was that it was horribly poorly compatible because most people never ran any of the test suites that Kronos charges for.  Take with a huge grain of salt as this is second-hand and wasn't the primary topic.

It would certainly be desireable that there is an standardised API that everyone uses to get access to embedded hardware codec support.

Always a good thing, if it's done well.


From: rtcweb-bounces@ietf.org<mailto:rtcweb-bounces@ietf.org> [mailto:rtcweb-bounces@ietf.org] On Behalf Of Harald Alvestrand
Sent: 03 November 2013 18:10
To: rtcweb@ietf.org<mailto:rtcweb@ietf.org>
Subject: [rtcweb] API standardization on phones? (Re: Platforms that support H264)

On 11/03/2013 06:52 PM, DRAGE, Keith (Keith) wrote:
And this is where it would be useful to see some work done, although probably not by IETF.

There a a few bits of API work out there, but nowhere do we see much push for adoption. As fae as I see there is no political reason why they should not be adopted, or even a fresh effort adopted to create some new ones. The only thing seems to be lack of impetus.

What's the industry state on standardization of APIs for phones?

Most of what I see seems to be platform specific, with some exceptions (OpenGL / WebGL for graphics, OpenES for audio).

But it's not where I spend the most time looking; it would be good to have someone who knows this space summarize

Randell Jesup