Re: [media-types] [art] [dispatch] Status of Haptics I-D in DISPATCH?

[John C Klensin <[e-mail]>]

Hi, Ted,

Also inline, with much trimmed.  As a preview, I can, and
largely do, agree with your "core proposition".  I just question
whether this draft represents the level of maturity appropriate
for AD sponsorship [0], whether the IETF is ready and has the
energy and competence to do the work, and whether a top-level
media type is appropriate and/or under what conditions.   Based
on what I have seen so far, I think the answer to all three
questions is "no".  The answer to the first and third is
normally "WG".  And the answer to the second is "try to figure
out a different way to get the work done".   Details below.

--On Tuesday, May 4, 2021 10:06 +0100 Ted Hardie
<[e-mail]> wrote:

> Hi John,
> 
> Response in-line.
> 
> On Tue, May 4, 2021 at 4:47 AM John C Klensin
> <[e-mail]> wrote:
> ...

> Ted,
 
> I believe the issue here isn't the state of the
> standards-making in another body but the core proposition of
> the effort:  haptic signals are media. There are a lot of
> consequences of that core proposition, the most important of
> which for me are that these signals can be bound with other
> media into coherent multimedia resources and that we can build
> interoperable standards that allow both haptic media and these
> multimedia resources to be handled by any compliant systems
> that follow that standard.  (The other obvious way to model
> them, as application instructions, leaves us in a very
> different place).
> 
> If we agree with that core proposition, than incorporating
> haptic signals into the media types and other aspects of media
> processing that are under the aegis of the IETF is the right
> thing to do, and the time to start that work is now, when it
> is still possible to have the aspects of the system handled
> elsewhere adjusted more readily.  Waiting until that work is
> completed strikes me as what I grew up hearing called "the
> wrong mistake".

>From a very high-level view, I agree with the above.  However,
it does not follow from "incorporating haptic... into the media
types" that a top-level type is needed, nor that this I-D is a
necessary and sufficient proposal to do that.  As a specific
example, MPEG was incorporated as a subtype under "video" from
the very beginning of MIME multimedia work despite normally
including audio elements.  Its subsequent variants and offspring
have been allocated there too.  I observe that the work in other
SDOs seems to be done in MPEG-related groups: that does not mean
"haptics" should not be a top-level type, but it does not
strengthen the case.  Could this be done as an "application"
subtype with appropriate parameters?  Perhaps.  And that would
certainly be the default for haptics as soon as someone pointed
out that no video was needed.  

Keep in mind that the naming part of the media type registration
model is ultimately about the names by which things are called
within a set of protocols, rather than the things themselves, or
even about what people call them. If the proposal were, e.g., to
allocate application/haptics (or even video/haptics), I assume
the discussion would be mostly with Ned (and potentially Murray
and Alexey) as type reviewers and not on the ART or DISPATCH
lists.  It would be mostly about whether the definition in the
document (including whatever stable or unstable references it
pointed to) was adequate and not about a fundamental principle
about how media type names and registrations are organized.

I think that, in the long term, the IETF lives or dies based of
the quality of its work.  Getting into so much of a hurry about
this that we ignore fundamental considerations would be, IMO, a
different "wrong mistake".

Perhaps equally important to my reaction to your core principle,
I don't see anything in either the I-D or the discussion that
follows that asks or expects the IETF to get involved in the
actual definitions or protocols associated with haptics.  What I
get from the draft is that there is a new media thing here and
that the IETF is being asked at assign a label -- a media type,
but remembering that RFCs 2046 and 4289 define that as a
type/subtype combination [1], not necessary a top-level one
although that is what the I-D proposes.   If someone came to us
and said "we'd like the IETF to be involved in designing how
haptics work, how the data structure for transmitting haptic
information is to be defined and organized, and/or how new
protocols for transmitting and presenting haptic information (or
delivering vibrations to your fingers or seat) are defined, than
we would be having a different conversation (from my point of
view, even more about expertise in the IETF).  But, as far as I
can tell, we have not been offered that opportunity and neither
MIME, nor the web, nor any other application that uses media
types care very much (except for defaults with some known types
and unknown subtypes used with them) what things are called,
only how the data are organized and whether that organization is
well-defined and implemented.   If you, or someone else, want to
make a proposal that we be involved that way, possibly in
competition with other groups, by all means write it.

But, as I stands, I see only a request for a name and so I would
suggest that the authors (and the IETF might reasonably choose
among the following:

(1) Go with application/haptics (or video/haptics but I'm
personally convinced by the authors' argument about that)  and
define it now, in a way that is satisfactory to the media-types
list and the reviewers.

(2) Make a deal with the reviewers that the subtype should be
registered new with an informal and incomplete definition but
that a more complete definition will come along when the work in
MPEG and ISO get further along.  The current rules don't
encourage that (for good reason) but don't explicitly prohibit
it either... and it might be The Right Thing to do for the
reasons you indicate.

(3) Insist that a top-level type is needed, but accept the fact
that such requests are, and should be, subject to very careful
scrutiny, scrutiny that necessarily include a review of what we
consider appropriate for a top-level type for which there is not
already significant experience and very stable specs (it is not,
e.g., a situation akin to the request for "font/").

The first should be relatively quick and focus on adequacy of
definition rather than question of whether haptic or
haptic-related data are appropriate for some sort of media type
registration (which, IMO, has never been in doubt-- again
agreeing with your core proposition).  The second, even quicker,
especially if it comes via a formal request from another
recognized standards organization (then the ADs would not even
need to be involved except maybe for another round on
"recognized").  If the third is really the preference, then a WG
to examine the issues, the criteria, and probably even the
merits of this proposal is the price one pays.  To which the
request remains one of a new top-level type, the time that might
take and the tradeoffs involved are really the authors' issue
and problem, not the IETF's.

> We could also, of course, ignore the work and that
> proposition.  The likely outcomes of that, in my hazy crystal
> ball, is either that the relevant work will create a MIME-like
> system that looks like but isn't quite actually MIME (which
> means that all of the other media types which are consumed by
> that system will have to deal with being part of two subtly
> different systems) or that deployed systems will squat on a
> string in a way that simply routes around our registry and
> rules.  I am sure you can supply the examples which lead me to
> that conclusion pretty readily.

I can indeed, but I think you are presuming more involvement by
the IETF in the definition of data structures and protocols than
I think has been asked for or offered (again as a handy example,
note that the IETF had zero involvement in defining MPEG itself
-- we just assigned a label to an existing spec -- and there is
no evidence that either MIME or the Internet have been worse for
the experience).  Now, if there were something in this proposal
(or some proposal that would likely emerge without IETF
involvement) that required a different type naming structure or
syntax or that created a subtype of, say, "text/", that would be
a problem, not just for MIME but for everything that now depends
on MIME structures that someone might want to adapt in the
future for haptics.  So, yes, we should facilitate their getting
a name and should, if asked, advise against stupidity.  But,
beyond that, the only thing MIME --as a set of structures and
principles-- cares about is that a stream of bits are
transmitted in some contained way.   I suppose that, if the data
to be transmitted came in Qbits, Frozzles, or some sort of three
or four dimensional data arrangement that could not be reified
into a stream, there would be a problem in which it would be
very important for the IETF to be involved.   But not only is
that not being proposed by this I-D but it is certainly nothing
we'd want to hurry through.
 
> I have not replied to your proposal below to cede the
> territory to another body, simply because I think a major
> point of any effort in this space is how multimedia works
> rather than how single-medium signals are consumed. That being
> the case, I don't think we can cede without ceding much more of
> the multimedia landscape than I think you intended.

I agree with that as a principle, but I don't see a proposal yet
-- from you, the authors (in the I-D or otherwise), or anyone
else, to get the IETF involved in the relevant protocol
definitions or the haptics part of "how multimedia works".  From
that point of view, the issue is not ceding the territory to
others but having it brought to us, pre-ceded, with what it
really just a request for a name (again, not that much different
from, e.g., what became video/H263-2000 or video/mp4).  I can
see a very strong case for the IETF being more actively involved
in how multimedia works, or possibly in producing a document
like RFC 6416 (which defined MP4V-ES), but the current I-D-
isn't any of those things and, AFAICT, we have no proposal, or
even the beginning of a proposal, in front of us for that sort
of work. 

> Just my opinion, of course.

And mine.

best,
   john

[0] I don't know that it has ever been formalized and don't know
that trying to do so would be a good idea, but one working
definition of "appropriate for AD sponsorship" includes at least
one of two elements.  One is that it is important to be
published in the IETF Stream, but no one other than the authors
really care or it is presented as their opinion and the impact
on the rest of the Internet is likely to be minimal.  The other
is that there are no controversial issues, especially those of
the variety that sorting out during IETF Last Call does not work
well or provide confidence about informed IETF consensus.  The
authors' arguments in the draft and yours about impact on MIEM
argue against the first case.   I think the question of whether
this, as defined in the I-D should be a top-level type is
clearly controversial even if the question of whether some name
should be assigned is not. 

[1] That terminology may have been a mistake and/or
confusion-prone, but sorting it out is just another argument for
the WG Ned suggested.