[Gen-art] Gen-ART Telechat review of draft-alakuijala-brotli-09

[Paul Kyzivat <[e-mail]>]

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed by the
IESG for the IETF Chair. Please wait for direction from your document
shepherd or AD before posting a new version of the draft. For more
information, please see the FAQ at <​ 
http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Document: draft-alakuijala-brotli-09
Reviewer: Paul Kyzivat
Review Date: 2016-04-28
IETF LC End Date: 2016-04-08
IESG Telechat date: 2016-05-05

Summary:

This draft has serious issues, described in the review, and needs to be 
rethought.

(Note: I first wrote this against -08 for an earlier telechat date. But 
it was rescheduled and -09 was released, so I'm resending it. I reviewed 
the changes between -08 and -09, and they don't affect my review.)

Major: 2
Minor: 0
Nits:  0

==========

(1) Major:

It is unclear to me whether this document seeks to be an authoritative 
specification of the Brotli format, or simply an explanation of the 
format the supplements another specification.

Evidence that it intends to be normative:

- The introduction states: "The purpose of this specification is to 
define a lossless compressed data format..." and "This specification is 
intended for use by software implementers to compress data into and/or 
decompress data from the brotli format."

- I had some discussion with the author after my Last Call review. In 
that discussion I learned of three successful attempts to create new 
implementations of Brotli using earlier versions of this document, 
without reference to another implementation.

Evidence that it doesn't intend to be normative:

- The intended status is Informational.

- There is no 2119 or similar language in the document

- The document provides a URL on github for an open source 
implementation. (However this is not a normative reference. There is no 
Reference section in the document.)

If this document is intended to be normative, then it should be 
restructured as such, using 2119 language, with actual references to 
stable documents, and with more clarity.

If this document is intended to be informative, then it should be 
restructured to formally identify the normative specification of the 
format, and then concentrate on supplementing that document with further 
explanation.

(2) Major:

If this document intends to be normative, *I* don't think it is 
sufficient for the task. I said the same in my Last Call review. I 
subsequently learned from the author that others have successfully used 
to create implementations, so perhaps my judgement is too harsh, but I 
remain concerned. I was told that an implementer spent 20-40 hours 
understanding this document sufficiently to create a decoder. (Distinct 
from the time required to do the implementation.) Perhaps the format is 
inherently difficult and such level of effort to understand it is 
expected. That is more effort than I was prepared to exert for a Gen-ART 
review. But without doing that I can't suggest how to improve the 
description.

The following is from my Last Call review:

I spent many hours trying to decipher the document, but ultimately 
failed. (I have no experience in implementing compression/decompression 
algorithms, but I have many decades of programming experience including 
that involving detailed bit manipulation and data representation.)

If two expert programmers were asked to independently build 
encoder/decoders in accord with this document, IMO it would be 
incredibly unlikely (impossible) that the resulting programs would be 
interoperable. And given the complexity of the encoding it would also be 
extremely difficult to figure out *why* they didn't interoperate.

My sense from reading this document is that the format is operationally 
defined by an existing program (https://github.com/google/brotli) and 
that this document is an attempt to re-render the source code in 
English. However the algorithms being described are so complex that 
English (at least *this* English) is inadequate for the job.

I started out making notes about particular places where I found the 
language confusing or ambiguous. But there were so many that I 
ultimately gave up.

I have serious doubts that the goal of this document achievable using 
natural language text. I don't have much of an idea of what to try to 
achieve this. It will take much more than just patching the current 
document. If possible at all it will require a vastly different approach.

To achieve the goal of having a specification independent of a 
particular implementation it may be necessary to abandon backward 
compatibility with *this* implementation. (I recognize that doing so may 
be unacceptable.)