Re: [mile] Benjamin Kaduk's Discuss on draft-ietf-mile-jsoniodef-10: (with DISCUSS and COMMENT)

["Takeshi Takahashi" <[e-mail]>]

Hi Benjamin,

Thank you very much for your kind replies.
Let us reply your comments as follows.

Below are the current version of the draft that reflected your kind comments.
https://github.com/milewg/draft-ietf-mile-jsoniodef

> We use a subset of the JSON "number" type to represent integers, which inherits JSON's range limits on numbers.  My understanding is that such limits are not present in IODEF XML (e.g., we do not specify a totalDigits value), so this is a new limitation of the JSON format that needs to be documented (and, technically, drops us out of full parity with the XML form).

We were not aware of these limitations.
So we have checked the web: http://json.org/, but we could not find any sentences that limit the limitation of numbers.
In CBOR, we cope with bignum in our draft, and the byte length of bignum is unlimited, in our understanding.

---------------------------
2.4.2.  Bignums

   Bignums are integers that do not fit into the basic integer
   representations provided by major types 0 and 1.  They are encoded as
   a byte string data item, which is interpreted as an unsigned integer
   n in network byte order.  For tag value 2, the value of the bignum is
   n.  For tag value 3, the value of the bignum is -1 - n.  Decoders
   that understand these tags MUST be able to decode bignums that have
   leading zeroes.

   For example, the number 18446744073709551616 (2**64) is represented
   as 0b110_00010 (major type 6, tag 2), followed by 0b010_01001 (major
   type 2, length 9), followed by 0x010000000000000000 (one byte 0x01
   and eight bytes 0x00).  In hexadecimal:

   C2                        -- Tag 2
      29                     -- Byte string of length 9
         010000000000000000  -- Bytes content
---------------------------

So, we are not sure whether we are pushing any limitation on the numbers in our draft.
We would appreciate any further information on this matter.

> The JSON "examples" seem to be using a "//" notation for comments, that is not valid JSON nor described by draft-zyp-json-schema, thus appearing to make the examples malformed (absent some other disclaimer of the commenting convention).

Agreed. The latest version of the draft has the following notes to cope with this comment.
'Note that in figures throughout this document, some supplementary information follows "#", but these are not valid syntax in JSON, but are intended to facilitate reader understanding.'

> How does STRUCTUREDINFO relate to EXTENSION?  What makes one vs the other appropriate for a given piece of information?  Since the former is only in  RFC 7203 and not 7970, we do not have an easy reference for their interplay, given 7970's minimal discussion of the use of 7203. (It sounds like STRUCTUREDINFO is for structures from other published specifications and EXTENSION is for more local/custom things, but I'm not entirely sure if that's exactly the intended split.)

IODEF version 1 had several extension mechanisms, using AdditionalData class (EXTENSION type).
RFC 7203 specifies one structured use of AdditionalData so that it facilitates to embed XML data.
RFC 7970 defines version 2 of the extension, but we thought defining next (independent) version of RFC7203 does not make sense and included the context inside the draft.
In the xml version of the STRUCTUREDINFO indicates the schema/structure of the content, thus the receiver may validate the content, while AdditionalData could be used to fit any types of data.
If receiver does not care about the structure of the data, one may not have to use STRUCTUREDINFO.

> It's somewhat surprising to see CBOR used but with CBOR maps required to use string form for representing map keys (i.e., no short integer key values are defined).
> Some of the strings that are map keys in the JSON objects are fairly long; is this extra space not a concern for the overall CBOR encoded document (e.g., due to containing a large quantity of binary data such that encoding overhead is a small relative portion of the encoded document)?

Thank you for pointing this issue.
We agree. Currently, we have not been sticking to shortening the byte sizes.
If it is necessary, we can define the map keys in the JSON objects, but we are not sure whether we want to define it here.

> I guess that since it seems to only be used in (non-normative) Appendix B, [jsonschema] can remain as an informative reference, though it would be nice to have a citation where it is actually used in the document, as opposed to just in the Introduction. 
> Since it is an informative reference, the following point is not Discuss-worthy: The current citation gives no absolute locator, leaving me somewhat unclear about whether to consult draft-zyp-json-schema or something on json-schema.org or some other source. 
> The contents of Appendix B suggest it is the second of those...

We agree, we should put the reference here.
As you guessed, we followed json-schema.org.
Therefore we put the reference to json-schema.org in Appendix B.

> Section 1

>    processing is JSON.  To facilitate the automation of incident
>    response operations, IODEF documents should support JSON
>    representation.
>
> Is it documents or implementations that should support the JSON representation?

We hope IODEF as a specification should support JSON representation.
Therefore, we hope to say that IODEF documents should support the JSON representation.
Having said that, implementation also needs to support JSON representation.
Therefore, let us rephrase so that IODEF documents and implementations should support the JSON representation.

> Section 2.1
>
> The string "STRUCTUREDINFO" does not appear in RFC 7203, so I think we need some additional locator information to indicate what behavior we're referring to.

Thank you very much for pointing this here.
We have added the following sentence: "Note that this type was originally specified in Section 4.4 of <xref target="RFC7203" /> as a basic structure of its extension classes"

> Using CBOR major type 2 for HEXBIN implies that actual binary values are recorded directly, i.e., without any "hex" encoding.  We should be clear about this one way or the other, and I didn't really see anything in the CDDL schema that called this out.

Figure 1 of the draft has the following information.

```txt
 | BYTE            | Section 2.5.1     | "string" per [RFC8259]        |
 | BYTE[]          | Section 2.5.1     | "string" per [RFC8259]        |
 | HEXBIN          | Section 2.5.2     | "string" per [RFC8259]        |
 | HEXBIN[]        | Section 2.5.2     | "string" per [RFC8259]        |
```

If we see the reference, it says as follows.

```txt
2.5.2.  Hexadecimal Bytes
​
   A binary octet encoded as a character tuple consistent of two
   hexadecimal digits is represented in the information model by the
   HEXBIN data type.  A sequence of these octets is of the HEXBIN[] data
   type.
​
   The HEXBIN and HEXBIN[] data types are implemented in the data model
   as an "xs:hexBinary" type per Section 3.2.15 of [W3C.SCHEMA.DTYPES].
```

So, if readers follow the reference, readers can reach the information.
We are not sure whether we should put more information here.
If you think it is better, we will add some sentences here, we rather prefer to keep it simple (because the draft is already too long).

> Section 2.2.2
>
> It's not claer to mey why using a plain text string is allowed for representing a ML_STRING, as on the face of it that could lose language information. 
> Is the idea that this is supposed to inherit from some higher-level element, or just to reflect an efficiency of encoding when neither of the optional language/translation-id fields are present?
> Regardless, we should be more clear about that, since neither here nor Section 5 includes any discussion thereof.  I see that Section 3.2 does have a brief statement about this being a change from RFC 7970, but I'd still like to see a little more clarity on this point.

Yes, as you pointed out, it is just for efficiency of encoding.
Though multilingual support is very appreciated and necessary, many information will still use only alphabets.
So, for those information that require multilingual support can use ML_STRING while those that does not require can use STRING.

In the XML version (RFC7970), it is easy to cope with alphabet-only string efficiently because xml:lang and translation-id fields are optional in the iodef:MLStringType (See section 2.4 of RFC7970).
"<Description lang=en>This is a sample<Description>" could be written as "<Description>This is a sample<Description>"
To realize the same efficiency for those alphabet-only string, we prepared ML_STRING and STRING.

To describe this issue, we have the following sentence in Section 3.2: "The elements of ML_STRING type in XML IODEF document are presented as either STRING type or ML_STRING type in JSON IODEF document."

>    Examples are shown below.
>    "MLStringType": {
>      "value": "free-form text",                              //STRING
>      "lang": "en",                                             //ENUM
>      "translation-id": "jp2en0023"                           //STRING
>    }

> That looks more like a schema than an example (especially with those //-comments!).  Also, nit-level, but there's only one, so "examples"
> plural does not apply.

Thank you, we've fixed this point: "An example is shown below"

> Section 2.2.4

>    SoftwareReference class is a reference to a particular version of
>    software.  Examples are shown below.
>
> "class" seems to be the prevailing RFC 7970 terminology but is not used much in this document; we seem to use "type" instead.

"type" would work as well.
However, since the caption of Figure 3 is "IODEF classes", and since we refer to RFC7970, we prefer to keep using the term "class" here, if it is not a big problem for you.

>    "SoftwareReference": {
>      "value": "cpe:/a:google:chrome:59.0.3071.115",    //STRING
>      "spec-name": "cpe",                                 //ENUM

> RFC 7970 suggests that the value portion is interpreted solely within the context defined by the "spec-name", so it's unclear to me if the initial "cpe:" prefix in this example is representative.

As you pointed out, "cpe:/" could not be necessary since the spec-name indicates that it will be a CPE-ID.
However, the CPE spec defines that the URI form of CPE-ID begins with cpe:/.
https://cpe.mitre.org/specification/

> Section 2.2.5

> I'd suggest to add a sentence along the lines of "Note that the structure of this information is not interpreted in the IODEF JSON, and the word 'structured' indicates that the data item has internal structure that is intended to be processed outside of the IODEF framework.

Thank you very much. We'll have added the sentence.

>    When embedding the raw data, base64 encoding defined in Section 4 of
>    [RFC4648] SHOULD be used for encoding the data, as shown below.

> Does this apply just to JSON or to CBOR as well?  I'm not sure if the CBOR HEX encoding actually uses raw binary or not, which would be more compact (and more recommendable?).

In Section 3.2, we have the following sentence.
```txt
   o  Signature, X509Data, and RawData are encoded with base64 and are
      represented as string (BYTE type) in JSON IODEF documents.
```
So, basically, yes, this applies to both JSON and CBOR.

> Section 3.2

>    o  Attributes and elements of each class in XML IODEF document are
>       both presented as JSON attributes in JSON IODEF document, and the
>       order of their appearances is ignored.

> Are there any practical consequences of this loss of information that we should discuss?

We do not think so, and we believe this loss is not important.
Indeed, I am not that sure whether the order of the appearance of attributes and elements is important even for RFC7970.

>    o  The elements of ML_STRING type in XML IODEF document are presented
>       as either STRING type or ML_STRING type in JSON IODEF document.
> Why?

As we discussed above, many of the documents still uses only alphabets, and RFC7970 can concisely represent those (thanks to the nature of XML). Because forcing to encode everyting in ML_STRING in JSON is not that efficient, we allowed to describe information in STRING as well.

> Section 5
>    SpecID = "urn:ietf:params:xml:ns:mile:mmdef:1.2" /  "private"
> This enum is managed by IANA; shouldn't we have some sort of signal in the CDDL to indicate it is extensible?  (Applies to any enum maintained by IANA.)

Yes, we agree.
We have put the following sentence in Section 3.2: "ENUM values in this document is extensible and is managed by IANA, as with <xref target="RFC7970" />."
(Since this document defines only the mapping to JSON, the value itself should be managed by RFC7970. Therefore, this draft does not have any IANA section.)

>    PortlistType = text .regexp "\\d+(\\-\\d+)?(,\\d+(\\-\\d+)?)*"
> \d matches by unicode properties; I think that we want just [0-9] here?

Thank you for pointing this issue.
We agree, we will change it accordingly.
(we haven't reflected this part yet, let us spent a bit more time to double check the data model.)

> Section 6
> I think we are making use of some fields whose content is controlled by IANA, so we may wish to consider asking IANA to update the description of the registry(ies) in question to include the JSON and CBOR usage.

That is one way, but we rather hope to keep it as a mapping to the RFC7970.
Therefore, if RFC7970 requests to change IANA tables, the users of this document can also access to the revised IANA table.

> Appendix B

> Is it more useful to have a non-normative JSON schema in the document, or a pointer to a tool that can generate one from the CDDL?
> (Alternately, how was this one generated -- were there any manual modifications needed from the output of some tool?)

These schema is manually made this time.
Currently, the non-normative JSON schema for IODEF is in the appendix.
Would you mean to move it to the main document?

Thank you very much, and best regards,
Takeshi Takahashi


-----Original Message-----
From: Takeshi Takahashi <[e-mail]> 
Sent: Monday, November 18, 2019 8:41 PM
To: 'Benjamin Kaduk' <[e-mail]>; 'The IESG' <[e-mail]>
Cc: '[e-mail]' <[e-mail]>; '[e-mail]' <[e-mail]>; '[e-mail]' <[e-mail]>
Subject: RE: [mile] Benjamin Kaduk's Discuss on draft-ietf-mile-jsoniodef-10: (with DISCUSS and COMMENT)

Hi Benjamin,

Thank you very much for your kind review, and I am sorry for not being able to reply you earlier.
Though we have submitted the revised version some time ago, I was unable to cope with your comments yet.
Let us reply to your points when I submit the next version.

Thank you, and best regards,
Take

PS: note that the latest version is available here: https://datatracker.ietf.org/doc/draft-ietf-mile-jsoniodef/


-----Original Message-----
From: mile <[e-mail]> On Behalf Of Benjamin Kaduk via Datatracker
Sent: Wednesday, September 4, 2019 4:09 PM
To: The IESG <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]
Subject: [mile] Benjamin Kaduk's Discuss on draft-ietf-mile-jsoniodef-10: (with DISCUSS and COMMENT)

Benjamin Kaduk has entered the following ballot position for
draft-ietf-mile-jsoniodef-10: Discuss

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-mile-jsoniodef/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

We use a subset of the JSON "number" type to represent integers, which inherits JSON's range limits on numbers.  My understanding is that such limits are not present in IODEF XML (e.g., we do not specify a totalDigits value), so this is a new limitation of the JSON format that needs to be documented (and, technically, drops us out of full parity with the XML form).

The JSON "examples" seem to be using a "//" notation for comments, that is not valid JSON nor described by draft-zyp-json-schema, thus appearing to make the examples malformed (absent some other disclaimer of the commenting convention).

How does STRUCTUREDINFO relate to EXTENSION?  What makes one vs the other appropriate for a given piece of information?  Since the former is only in  RFC 7203 and not 7970, we do not have an easy reference for their interplay, given 7970's minimal discussion of the use of 7203.
(It sounds like STRUCTUREDINFO is for structures from other published specifications and EXTENSION is for more local/custom things, but I'm not entirely sure if that's exactly the intended split.)

Can the shepherd please report on what level of validation has occurred on the CDDL syntax, the mappings between RFC 7970's content and this document's content, and the consistency between the formal syntax and the body text (e.g., listings of enum values, member fields of each type, etc.)?


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

It's somewhat surprising to see CBOR used but with CBOR maps required to use string form for representing map keys (i.e., no short integer key values are defined).  Some of the strings that are map keys in the JSON objects are fairly long; is this extra space not a concern for the overall CBOR encoded document (e.g., due to containing a large quantity of binary data such that encoding overhead is a small relative portion of the encoded document)?

I guess that since it seems to only be used in (non-normative) Appendix B, [jsonschema] can remain as an informative reference, though it would be nice to have a citation where it is actually used in the document, as opposed to just in the Introduction.  Since it is an informative reference, the following point is not Discuss-worthy: The current citation gives no absolute locator, leaving me somewhat unclear about whether to consult draft-zyp-json-schema or something on json-schema.org or some other source.  The contents of Appendix B suggest it is the second of those...

Section 1

   processing is JSON.  To facilitate the automation of incident
   response operations, IODEF documents should support JSON
   representation.

Is it documents or implementations that should support the JSON representation?

Section 2.1

The string "STRUCTUREDINFO" does not appear in RFC 7203, so I think we need some additional locator information to indicate what behavior we're referring to.

Using CBOR major type 2 for HEXBIN implies that actual binary values are recorded directly, i.e., without any "hex" encoding.  We should be clear about this one way or the other, and I didn't really see anything in the CDDL schema that called this out.

Section 2.2.2

It's not claer to mey why using a plain text string is allowed for representing a ML_STRING, as on the face of it that could lose language information.  Is the idea that this is supposed to inherit from some higher-level element, or just to reflect an efficiency of encoding when neither of the optional language/translation-id fields are present?
Regardless, we should be more clear about that, since neither here nor Section 5 includes any discussion thereof.  I see that Section 3.2 does have a brief statement about this being a change from RFC 7970, but I'd still like to see a little more clarity on this point.

   Examples are shown below.

   "MLStringType": {
     "value": "free-form text",                              //STRING
     "lang": "en",                                             //ENUM
     "translation-id": "jp2en0023"                           //STRING
   }

That looks more like a schema than an example (especially with those //-comments!).  Also, nit-level, but there's only one, so "examples"
plural does not apply.

Section 2.2.4

   SoftwareReference class is a reference to a particular version of
   software.  Examples are shown below.

"class" seems to be the prevailing RFC 7970 terminology but is not used much in this document; we seem to use "type" instead.

   "SoftwareReference": {
     "value": "cpe:/a:google:chrome:59.0.3071.115",    //STRING
     "spec-name": "cpe",                                 //ENUM

RFC 7970 suggests that the value portion is interpreted solely within the context defined by the "spec-name", so it's unclear to me if the initial "cpe:" prefix in this example is representative.

Section 2.2.5

I'd suggest to add a sentence along the lines of "Note that the structure of this information is not interpreted in the IODEF JSON, and the word 'structured' indicates that the data item has internal structure that is intended to be processed outside of the IODEF framework.

   When embedding the raw data, base64 encoding defined in Section 4 of
   [RFC4648] SHOULD be used for encoding the data, as shown below.

Does this apply just to JSON or to CBOR as well?  I'm not sure if the CBOR HEX encoding actually uses raw binary or not, which would be more compact (and more recommendable?).

Section 3.1

[[I stopped verifying mappings at DetectionPattern]]

Section 3.2

   o  Attributes and elements of each class in XML IODEF document are
      both presented as JSON attributes in JSON IODEF document, and the
      order of their appearances is ignored.

Are there any practical consequences of this loss of information that we should discuss?

   o  The elements of ML_STRING type in XML IODEF document are presented
      as either STRING type or ML_STRING type in JSON IODEF document.

Why?

Section 5

   SpecID = "urn:ietf:params:xml:ns:mile:mmdef:1.2" /  "private"

This enum is managed by IANA; shouldn't we have some sort of signal in the CDDL to indicate it is extensible?  (Applies to any enum maintained by IANA.)

   PortlistType = text .regexp "\\d+(\\-\\d+)?(,\\d+(\\-\\d+)?)*"

\d matches by unicode properties; I think that we want just [0-9] here?

Section 6

I think we are making use of some fields whose content is controlled by IANA, so we may wish to consider asking IANA to update the description of the registry(ies) in question to include the JSON and CBOR usage.

Appendix B

Is it more useful to have a non-normative JSON schema in the document, or a pointer to a tool that can generate one from the CDDL?
(Alternately, how was this one generated -- were there any manual modifications needed from the output of some tool?)


_______________________________________________
mile mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/mile