Re: [Cellar] Second AD review of draft-ietf-cellar-ebml-10

[Dave Rice <[e-mail]>]

Hi Alexey,

> On Oct 24, 2019, at 8:33 AM, Alexey Melnikov <[e-mail]> wrote:
> 
> Hi Dave,
> 
> On Thu, Jul 11, 2019, at 2:53 PM, Dave Rice wrote:
>>> On Jul 11, 2019, at 9:36 AM, Alexey Melnikov <[e-mail] <mailto:[e-mail]>> wrote:
>>> 
>>> Hi Dave,
>>> I removed comments where we are in agreement. A few followups:
>>> 
>>> On Sat, Jul 6, 2019, at 6:23 PM, Dave Rice wrote:
>>>> Hello Alexey,
>>>> 
>>>> Thanks much for providing this thorough review. I’ll try to reply point by point below with references to either pull requests or issues or offer followup questions. In a few places I ping Steve Lhomme and Michael Richardson as the comments relate to text they originated in their work.
>>>> 
>>>>> EBMLElementOccurrence    = [EBMLMinOccurrence] "*" [EBMLMaxOccurrence]
>>>>> EBMLMinOccurrence        = 1*DIGIT
>>>>> EBMLMaxOccurrence        = 1*DIGIT
>>>>> 
>>>>> Are there any upper limits on allowed values for these fields? Even if you don't encode them using ABNF, it would be good to mention them in an ABNF comment.
>>>>> 
>>>>> VariableParentOccurrence = [PathMinOccurrence] "*" [PathMaxOccurrence]
>>>>> PathMinOccurrence        = 1*DIGIT
>>>>> PathMaxOccurrence        = 1*DIGIT
>>>>> 
>>>>> Same comment as above.
>>>> 
>>>> I looked for examples of that sort of commenting but didn’t find much guidance. Eventually I simply appended " ; no upper limit” to each of the four referenced lines and added that to https://github.com/cellar-wg/ebml-specification/pull/265 <https://github.com/cellar-wg/ebml-specification/pull/265>.
>>> I think this is the wrong fix. Is it sufficient for an implementation to use 32 bit value to represent any of these? 64 bit value?
>>> "no upper limit" is not going to be interoperable.
>> 
>> The smallest possible EBML Element is 2 bytes (1 bytes Element ID, 1 byte Element Data Size, and 0 bytes of Element Data). The upper limit of how many occurrences would be determined by the limit of the Element Data Size of the Parent Element. If the EBML Document has EBMLMaxSizeLength as the default of 8, then the upper limit of an Element Data Size is 72,057,594,037,927,934 bytes. Is the smallest possible Element is 2 bytes, then the upper limit of that Element’s occurrence would be 72,057,594,037,927,934 divided by two.
>> 
>> So if EBMLMaxSizeLength = 8, then this is possible
>> 
>> <RootElement>
>> <TwoByteElement/>  # 36,028,797,018,963,967 occurrences
>> </RootElement>
>> 
>> but this would overflow the Element Data Size of the Root Element
>> <RootElement>
>> <TwoByteElement/>  # 36,028,797,018,963,968 occurrences
>> </RootElement>
>> 
>> However, EBML allows the EBML Document Type to set an EBMLMaxSizeLength value higher than 8, and as that is incremented up the upper limit would expand exponentially.
> I will try one more time: if I write an implementation that has hardcoded limit of 8 would it be considered compliant with this specification. I think asking to support arbitrary length values is a big ask.

Matroska, as a document type of EBML, does this. The draft for Matroska has a section “Added Constraints on EBML” that limits EBMLMaxSizeLength to a range of 1-8 at https://tools.ietf.org/html/draft-ietf-cellar-matroska-03#section-6.1 <https://tools.ietf.org/html/draft-ietf-cellar-matroska-03#section-6.1> and that is reiterated in the EBML Schema for Matroska at https://tools.ietf.org/html/draft-ietf-cellar-matroska-03#section-9.2 <https://tools.ietf.org/html/draft-ietf-cellar-matroska-03#section-9.2>.

Though EBML does not limit the EBMLMaxSizeLength, the default value is 8 (see https://www.ietf.org/id/draft-ietf-cellar-ebml-13.html#name-ebmlmaxsizelength-element <https://www.ietf.org/id/draft-ietf-cellar-ebml-13.html#name-ebmlmaxsizelength-element>), so a document type definition would have to opt into a >8 value if they felt as if it was needed.

The fifth paragraph of https://www.ietf.org/id/draft-ietf-cellar-ebml-13.html#name-ebml-schema <https://www.ietf.org/id/draft-ietf-cellar-ebml-13.html#name-ebml-schema> starting with "An EBML Schema MAY constrain the use of EBML Header Elements” discusses the scenario of using a non-default value for EBMLMaxSizeLength in an EBML Schema that defines a EBML document type.

>> In my draft I would considering "no upper limit” and 36,028,797,018,963,967 to effectively be the same or that the definition does not define the limit but in practice it would be limited by the capacity of the Element Data Size of the parent Element.
> I will let the document start IETF LC without this issue being fully resolved, but I suspect it will come up again during IESG review.
> 
>>>>> 11.1.10.1.  label
>>>>> 
>>>>>   The label provides a concise expression for human consumption that
>>>>>   describes what the value of the "<enum>" represents.
>>>>> 
>>>>> Is it worth adding a cross reference to the "lang" attribute here?
>>>> 
>>>> Do you mean to express the language of the term used within the label? Currently the language of the label is undefined and since it is an attribute that label is not repeatable.
>>> 
>>> To be honest I am not yet sure how I feel about "undefined language" here. Need to think about that.
>>> But either way, I think adding some text that "lang" attribute doesn't apply would be helpful.
>> 
>> In the case of Matroska, the labels are in English. In the EBML definition we could say that the labels are in English unless the definition of that associated EBML Document Type claims otherwise.
> Ok.
> 
>>>>> 12.  Considerations for Reading EBML Data
>>>>> 
>>>>>   If a Master Element contains a CRC-32 Element that doesn't validate,
>>>>>   then the EBML Reader MAY ignore all contained data except for
>>>>>   Descendant Elements that contain their own valid CRC-32 Element.
>>>>> 
>>>>> I don't fully understand your use of "MAY ... except ..." here.
>>>>> Can you elaborate on why would an implementation ignore data contained in a Master Element and not ignore Descendant Elements, even if they own CRC-32 is valid?
>>>> 
>>>> For instance if a Matroska file has three metadata tags and each has a CRC value and so does the parent Tags element like this.
>>>> 
>>>> <Tags crc=invalid>
>>>>   <Tag crc=valid>
>>>>   <Tag crc=valid>
>>>>   <Tag crc=invalid>
>>>> <Tags>
>>>> 
>>>> We’re trying to say that even though the contents of the <Tags> element is invalid, that the valid child elements may still be used.
>>> So to me this means that after discarding all invalid elements you end up with something like this:
>>> 
>>> <Tags >
>>>   <Tag crc=valid>
>>> </Tags>
>>> 
>>> As this is an incomplete document, I am struggling to understand what it can be used for?
>> 
>> In that case, the valid Tags are still useful even if within a parent element whos contents are invalid. Perhaps another example would be in the Attachments Element:
>> 
>> <Attachments crc=invalid>
>> <Attachment crc=invalid>Poster Art</Attachment>
>> <Attachment crc=valid>Subtitle Font</Attachment>
>> </Attachments>
>> 
>> Here some bit damage occurs to the Poster Art so the Attachment and its parent Attachments now have invalid crcs, however the subtitle font is still ok and could be used in the presentation without an issue.
> I suggest some examples along the lines of your reply should be added to the document. This is not a property commonly found in IETF formats, so talking more about it would be useful.

I drafted an example at https://github.com/cellar-wg/ebml-specification/pull/296 <https://github.com/cellar-wg/ebml-specification/pull/296>.
Thanks much,
Dave Rice