Re: [Cellar] compression libraries in IETF specifications (was Re: [cellar-wg/matroska-specification] deprecate LZO compression (#376))

[Spencer Dawkins at IETF <[e-mail]>]

Hi, Michael,

On Sun, May 3, 2020 at 11:10 AM Michael Richardson <[e-mail]> wrote:

>
> Hi, I wanted to bring this item to the list's attention.
>
> Spencer: I seem to remember that the IETF has had compression IPR issues in
> specifications in the past.  IPCOMP comes to mind, but I am guessing that
> other RTP/SIP groups have gone through this..


I had forgotten IPCOMP, but was remembering RoHC and RoHCv2. I think you're
remembering SIGCOMP.

The complete list of disclosures that mention "compression" is at
https://datatracker.ietf.org/ipr/search/?draft=&rfc=&doctitle=compression&submit=doctitle&group=&holder=&iprtitle=&patent=


> Is there possibly an IESG
> statement that might help us here?
>

I believe the only relevant IESG statement on licensing for repositories is
to make it clear that they need to include standard boilerplate (
https://www.ietf.org/about/groups/iesg/statements/open-source-repositories-license/
).

>
> But, first, the software license on the libraries is not entirely germane.
> An IETF specification can well contain algorithms which are currently only
> available in software licensed in ways that might not be useable by all
> implementers.
> Someone just will have to write a new library with a license that suits
> them!
>
> What matters if that the specification (and any related patents!) is
> available to all.  The IETF's minimum IPR requirement is generally RAND,
> which is often not good enough for open source, which requires RF.
> The decision as to which IPR claims to tolerate is a WG-level decision,
> with
> the proviso that the WG (and IETF) is informed about the issue.
>
> So even in the absense of the jackoalan and zivillian LZO implementations,
> the WG should be free to standardize LZO, if it the IPR claims on the
> algorithm itself are well understood.
>

This is exactly right. The IESG cares that the working group made the
decision to request publication on a document without being "submarined" by
undisclosed patents or patients with vague or confusing licensing
provisions. If we have our eyes open, we'll be fine.


> {In the compression space, my understanding is that there have been formats
> where there were no IPR claims on decompressors, but there was significant
> IPR-encumbered innovation on doing compression.  An encoder might need a
> specific license in order to get 70% compression rather than just 40%, but
> this had no affect on the decoder.  But, I don't recall the specific
> situations where this occured.  Maybe back in the old ZOO/ARC/ZIP days...}
>

I have the same vague memories (perhaps I'm remembering that licensing was
different, rather than decompressors hat no known IPR, but that's pretty
close).

I hope this helps!

Spencer


> Steve Lhomme <[e-mail]> wrote:
>     > Matroska Blocks can be compressed with zlib, bz2 and lzo compression.
>
>     > zlib and bz2 have decompression libraries available under permisssive
>     > licenses (BSD like for bz2 and OSI approved, GPL compatible, non
>     > copyleft for zlib).
>
>     > lzo doesn't have such a permissive library. The [official
>     > library](https://www.oberhumer.com/opensource/lzo/) is GPLv2, there
> is
>     > also decompression code that doesn't rely on this library [in
>     > FFmpeg](https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/lzo.c
> )
>     > which is LGPLv2 which means grabbing the whole libavutil to compile.
>
>     > This license is not permissive enough to be used in many closed
> source
>     > products where they don't want to touch any GPL-based license. That
>     > means a file that is created with LZO in a GPL/LGPL software may not
> be
>     > read universally because of licensing issues. I think we should avoid
>     > it and mark LZO as deprecated.
>
>     > The LZO compression advantage over others is that it is [very fast to
>     > compress/decompress](
> https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO).
> But
>     > the compression ratio is not good. That is good for realtime
>     > transmission but bad for long term storage. In the end automatic
> tools
>     > that test each compression (like mkclean) will never pick it anyway.
>
> ....
>     > I found 2 other options with better licenses:
>
>     > - a [C++ version in MIT license](https://github.com/jackoalan/lzokay)
> with compression
>     > - a [C# version in MIT license](https://github.com/zivillian/lzo.net)
> without decompression
>
>     > They are both based on the [Linux Kernel documentation for
>     > LZO](
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/lzo.txt
> ).
>
>     > In libmatroska2 I've been using minilzo GPLv2 but I may try to port
> one
>     > of these in C so I can use it (or simply write a C wrapper around the
>     > C++ one). That means there would be an alternative that is MIT
> license,
>     > so freely used by anyone.
>