Re: [AVTCORE] My notes for https://datatracker.ietf.org/doc/html/draft-hsyang-avtcore-rtp-haptics-00

[hyunsikYang <[e-mail]>]

Dear Spencer & AVTCORE



How are you?

Thank you for taking the time to review our draft.



We have made revisions and submitted today based on your feedback.



Additionally, You can find the answer in the draft 01 version, but I also
provided a brief response in the email below for your convenience.



Thanks.

Best regards,



*From:* Spencer Dawkins at IETF <[e-mail]>
*Sent:* Tuesday, November 7, 2023 11:59 AM
*To:* Hyunsik Yang <[e-mail]>; Xavier De Foy <
[e-mail]>
*Cc:* IETF AVTCore WG <[e-mail]>
*Subject:* My notes for
https://datatracker.ietf.org/doc/html/draft-hsyang-avtcore-rtp-haptics-00



Hi, Hyunsik and Xavier, Thank you for submitting this draft to AVTCORE. I
do have some suggestions, but overall, I think it's quite good. and very
close to being adoption-ready. Most of my suggestions are opportunities for
clarifications. Please feel free to ask for
clarifica ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌



Hi, Hyunsik and Xavier,



Thank you for submitting this draft to AVTCORE. I do have some suggestions,
but overall, I think it's quite good. and very close to being
adoption-ready.



Most of my suggestions are opportunities for clarifications.



Please feel free to ask for clarifications, or reply, on the AVTCORE
mailing list.



Best,



Spencer



   - One major point, but fortunately, it's only a process point you're
   addressing early - this spec uses [ISO.IEC.23090-31], which is not freely
   downloadable by just anyone. The good news is that Stephan Wegner, the IETF
   liaison manager for JTC1/SC 29, has experience with obtaining copies of
   JTC1/SC 29 specifications for reviewers - he told me you should send him
   the FCD, and he will distribute it to only those IETF people who have need
   to know.



   - Nit: please expand acronyms on first use, especially in the Abstract,
   and especially "MIHS!"



*[HY] thanks. Done.*



   - Nit: please update the Conventions section to reflect BCP14, and
   include RFC2119 and RFC8174 as a Normative References, as



*[HY] Thanks. we updated the section.*



The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
[RFC2119] [RFC8174] when, and only when, they appear in all capitals, as
shown here.



[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels",
BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <
https://www.rfc-editor.org/rfc/rfc2119>.



[RFC8174]

Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP
14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <
https://www.rfc-editor.org/rfc/rfc8174>.



   - In Section 3, terminology, this might be clearer:



This document uses the definitions of the MPEG Haptics Coding standard
[ISO.IEC.23090-31]. Some terms from that document are provided here for
convenience.



*[HY] Thanks. we updated sentence.*





   - For Section 4.1, please add "hmpg" to the Definitions section, with
   its expansion



*[HY] Thanks. we added the definition in section 3.*



   - For Section 4.2, please add "independent", "sync", "dependent",
   "non-sync", "time-independent", and "time-dependent" to the Definitions
   section



*[HY] Added definitions and removed sync/non-sync from the text, since it
is redundant with the equivalent dependent/independent (we left
sync/non-sync in the definition of dependent/independent to facilitate the
link with the MPEG specifications, which use both dependent/independent and
sync/non-sync).*



   -  For Section 4.3, I'm puzzled by the use of SHOULD, for a new
   specification that hasn't been deployed. Upon reflection, I don't think the
   considerations in this section need to be normative at all. Here's my
   suggestion:



The following considerations apply for the streaming of MIHS over RTP:



If a media sender sets durations with the same value for all non-zero
duration MIHS units between initialization MIHS units, this will make the
decoder more robust to RTP packet loss.



If a media sender sends one, or a few, MIHS silent units at the beginning
of a haptic silence, and does not send subsequent consecutive silent units,
this will allow the sender and receiver to save network resources.



If a media receiver receives a MIHS silent unit, the receiver can assume
that silence is intended until the reception of a non-silent MIHS unit.
This will make the decoder more robust to RTP packet loss.



EDITOR'S NOTE: these considerations may need to be re-evaluated depending
on the finalization of the haptics coding specifications in MPEG.





*[HY] Thanks for your suggestion, we reworked the text of the section based
on your input (and incidentally we also moved the section in section 5
since section 4 is more of a background section). About the MAY/SHOULD
language, we updated the text and added some consideration to clarify the
tradeoffs.*



   - For Section 5.2, each of the field descriptions includes both the
   description of the field, and language about the way different values are
   processed by the receiver. I suggest moving the language about processing
   to section 5.3. So, for example,



D (Dependency, 1 bit): this field is used to indicate whether the MIHS unit
included in the RTP payload is, when its value is one, dependent (i.e.,
"non-sync") or, when its value is zero, independent (i.e., "sync"). In case
of congestion, a receiver or intermediate node MAY prioritize independent
packets over dependent ones, since the non reception of an independent MIHS
unit can prevent the decoding of multiple subsequent dependent MIHS units.



would be split into



D (Dependency, 1 bit): this field is used to indicate whether the MIHS unit
included in the RTP payload is, when its value is one, dependent (i.e.,
"non-sync") or, when its value is zero, independent (i.e., "sync").



in Section 5.2, and



In case of congestion, a receiver or intermediate node MAY prioritize
independent packets over dependent ones, since the non reception of an
independent MIHS unit can prevent the decoding of multiple subsequent
dependent MIHS units.



*[HY] Thank you, we moved the text on processing by the receiver in section
8, since this was about congestion control.*



   - In Section 5.3, there's a mention of "aggregation". This would be
   clearer if the text said



Editor's Note: consider if it would be useful to add the ability to
aggregate multiple MIHS units in a single RTP payload - for instance, to
aggregate multiple MIHS units with different layer values into a single RTP
payload.



*[HY] Thanks for your comment. We revised sentence based on your comment.*



   - In Section 5.3, there's a figure that contains the payload structure
   type values. It would be good to also provide a figure that contains the
   MIHS Layer values, with a pointer from the description in Section 5.2.



*[HY] The semantic of MIHS layer values is not specified, we added a
sentence in 5.2 (L field) to clarify this point (the application can use
layers as it sees fit, as long as it respects the priority order of the
layers).*



   - In Section 5.3.2, it would be good to point out that the use of MIHS
   Unit Fragmentation in RTP means that a media receiver can receive some
   fragments, but not other fragments, and the missing fragments would not be
   retransmitted by RTP. If there is any danger of an actuator behaving oddly
   because it interpreted a partial MIHS Unit Fragment, you might want to
   mention that, as guidance to implementers.



*[HY]  Thanks for your comment. We added a sentence based on your input in
section 5.3.2*



   - In Section 6, is it correct to say



"This section specifies the optional parameters.described in
[ISO.IEC.23090-31]"?



*[HY]  Yes. *





   - In Section 6.1, I suggest moving this text to the end of Section 6:

The receiver MUST ignore any parameter unspecified in this memo.



*[HY]  Thanks for your comment. We moved it to section 6.*



   - In Section 6.1, this looks a lot like an IANA Considerations section,
   but the information here is much abbreviated, compared to
   https://datatracker.ietf.org/doc/html/draft-ietf-mediaman-haptics-05#name-subtype-registrations.
   Since Section 10, IANA Considerations, just says "see Section 6", I'd
   suggest adding all of the attributes used in those registrations,
   especially since
   https://datatracker.ietf.org/doc/html/draft-ietf-mediaman-haptics-05#name-subtype-registrations
   says those registrations are also provided as examples. Alternatively, you
   could move the rest of Section 6 (including 6.1 and 6.2) into the IANA
   Considerations section, where IANA would be looking for what you want them
   to know, anyway.



*[HY] First of all, we agree that the media subtypes haptics/hmpg,
haptics/hjif, haptics/ivs will be registered with IANA through media man
draft, which also defines the haptics top-level media type. So, now, we
only have to update haptics/hmpg with IANA (i.e., to add optional
parameters) Therefore, we revised sections 10 and 6 + 6.1  to indicate this
is a registration update and point to the original registration.*



   - In Section 6.2 - I know this is going to be complicated, but I'll do
   my best. 🙃



My understanding - please correct me - is that most or all of these
optional parameters are described in [ISO.IEC.23090-31], including values,
and all that we're doing here is naming the parameters in a way that SDP
would understand. So I THINK all that's required in this document, is to
say (for example)



*Corresponding parameter in [ISO.IEC.23090-31]*

*SDP Parameter Name*

…

…

avatar type

hmpg-avtypes

…

…





*[HY] We totally agree with your opinion. We updated the text to refer to
the parameter name in [ISO.IEC.23090-31] and provided the valid values as
well like below.*



*hmpg-profile Indicates the profile used to generate the encoded stream as
defined in {{ISO.IEC.23090-31}}: MPEG_haptics object.profile is a string
which may in the initial release of the specifications hold the values
"simple-parametric" or "main".*



If the IETF will not define new SDP parameters without a change to
[ISO.IEC.23090-31], that should be about right. The other option would be
to include values for each SDP parameter name from [ISO.IEC.23090-31], for
the convenience of the reader.



If the IETF wants to use existing parameters that are already defined in
[ISO.IEC.23090-31], we can update this RFC, or we could create an IANA
registry for these parameters, with



Change controller: ISO/IEC JTC1/SC 29/WG 7 (MPEG 3D Graphics and Haptic
Coding)



*HOWEVER*, if the IETF wants to use *NEW *parameters that are defined in
revisions to [ISO.IEC.23090-31], using an IANA registry for these
parameters becomes much more appealing.



If the IETF ever decides to define new parameters that aren't already in a
JTC1/SC 29/WG 7 specification, using an IANA registry is almost certainly
the Right Thing To Do.



I note that our AD, Murray Kucherawy, is a backup Designated Expert for
https://www.iana.org/assignments/media-types/media-types.xhtml, and you're
already talking to Stephan Wegner, the IETF liaison manager for JTC1/SC 29
about availability of [ISO.IEC.23090-31] for reviewers, I'm sure AVTCORE
has the wisdom I lack, to work through all the details here!



*[HY] Thanks for bringing this up. We may discuss this topic in a future WG
meeting to better understand if a IANA registry is the right way to go. And
in reference to your mention of [ISO.IEC.23090-31] availability for
reviewers: yes, based on our exchange with Stephan, there should be a
version available to the WG (through a LS) at some point in February.*




[image: Banner]

[image: Banner]
<http://www.interdigital.com/white_papers/defining-the-xr-experience-enabling-the-immersivity-ecosystem-?utm_source=signature&utm_medium=Email&utm_term=xr&utm_content=banner&utm_campaign=defining_the_xr_experience>

Defining the XR Experience: Enabling the Immersivity Ecosystem
<http://www.interdigital.com/white_papers/defining-the-xr-experience-enabling-the-immersivity-ecosystem-?utm_source=signature&utm_medium=Email&utm_term=xr&utm_content=banner&utm_campaign=defining_the_xr_experience>

This e-mail is intended only for the use of the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential and/or otherwise protected from disclosure to anyone other
than its intended recipient. Unintended transmission shall not constitute
waiver of any privilege or confidentiality obligation. If you received this
communication in error, please do not review, copy or distribute it, notify
me immediately by email, and delete the original message and any
attachments. Unless expressly stated in this e-mail, nothing in this
message or any attachment should be construed as a digital or electronic
signature.