Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)

[Michael Slusarz <[e-mail]>]

Apologies on taking too long to respond.

I neglected to update the draft with the other changes discussed in previous messages; I will upload that now.

See below for additional responses.


> On June 2, 2019 11:33 AM Barry Leiba <[e-mail]> wrote:
> 
>  
> I think we're still in significant disagreement here; I'd like others
> from the working group to comment, and see if that can move us in one
> direction or the other.
> 
> > > > I'll start with providing a (future) real-world example, and why this
> > > > behavior exists in its present form.  Given a hypothetical future
> > > > algorithm "IMAGE" (generate image/jpeg preview if image data exists in
> > > > message), I issue this preview command:
> > > >
> > > > a FETCH 1 (PREVIEW (LAZY=IMAGE FUZZY))
> > > >
> > > > This is asking the server: provide me an image preview of the message,
> > > > but ONLY if 1) the message actually contains image information and 2)
> > > > that image information is already generated or can be generated very
> > > > quickly.  Otherwise, fall back to FUZZY generation.
> > >
> > > Hm.  That really seems like a contrived example.  How would the client
> > > software possibly know whether there's useful information in the
> > > image, or whether the information in the accompanying text is more or
> > > less important?  Suppose I have two messages:
> > >
> > > 1. text/plain that basically says nothing; image/jpeg that has a
> > > diagram that explains everything
> > >
> > > 2. text/plain that has the entire real content of the message;
> > > image/jpeg of the senders company logo
> > >
> > > It's pretty clear from that explanation that for message 1 I want some
> > > sort of description of the image and for message 2 I want a preview of
> > > the text.  But how would the client software know that, and,
> > > therefore, know what to ask for?
> >
> > Is that what a typical received message looks like?  To me, that looks like the edge case.
> 
> It's not the "typical" message, but I don't think it's an edge case
> either.  I do often get messages where the only significant content is
> in an image.  (I've tried to dissuade those responsible from sending
> things that way, but they persist.)  So, no, I wouldn't call it an
> "edge case" either.  And even if it were, that's not really the point;
> see below.

Agreed that this is not really the point of this discussion.  We aren't discussing how an image preview algorithm would work, since in practice it would likely need to be designed to be smart enough to not return image data for things like embedded company logos. (This is approaching "what is an attachment?" territory.)


> > Seems that a server could figure out (through experience) when image
> > preview data should be calculated or ignored.  Image size, MIME
> > structure information, MIME order, etc. could all be used to reach some
> > sort of 95% accuracy threshold.  Maybe that doesn't happen day 1 of
> > implementation, but a competitive, reactive server implementation will
> > work on this.
> >
> > The same issue exists for text previews, FWIW.  What about a text/plain
> > + text/html alternative message, where text/plain is "Please click on
> > this URL to view the text part", and the text/html part is empty text
> > (it's a single image link).  A server can be programmed to be smart
> > enough to ignore that part re: preview.
> >
> > As written, we allow a server to be given the latitude to figure these
> > kind of issues out.  Closing down that opportunity because it might be
> > a difficult logical problem isn't the right approach from my view.
> 
> I note that in your response you say "server" four times, at least
> once per paragraph, and "client" never.  But my point (see above) was,
> "But how would the client software know that, and, therefore, know
> what to ask for?"  These are things a *client* is requesting; how does
> the client know whether to request that the preview be preferentially
> done on the image or on the text?  I contend that it doesn't, and
> can't, and that, therefore, it's strange to expect it to ask, and
> strange to say that the server has to take the client's "priority
> decision" into account.

Client: "Please provide me the richest content you can - richest content is defined via the priority list I provided."

I don't see how a client providing a priority order of preview algorithms is any different than what the client would do if it had to download the message and parse it for preview data?

Also, how this is any different than issuing two consecutive FETCH PREVIEW commands?  If the first PREVIEW command returns the empty string, a client is just going to send another FETCH command using the next PREVIEW algorithm in the list.  Thus, the client is indicating its "priority decision" by the order it is sending its commands.  Why not do that in a single command?


> I'm still looking for a real use case where the *client* is the entity
> that's better suited to asking for any specific action with respect to
> a message preview.

See below.

 
> > > Apart from that, if we're going to do any sort of text-based preview
> > > of an image, I would think we'd just say that when FUZZY is applied to
> > > an image, that's what it does.
> >
> > FUZZY is explicitly defined as text only output though, since that's generally mapping
> > what clients already do.
> 
> Well, but saying, "That's the way it works now," isn't really the
> point.  We need to be looking at how we *want* this to work going
> forward, and I think that having the *server* decide what the best way
> is to give a preview of a message is the right answer.  And if that
> means that the server decides to make a preview from the text/plain
> part, the text/html part, the application/pdf part, or the image/jpeg
> part (perhaps using OCR on the image), then I think that's fine.
> FUZZY is explicitly defined as text only, only because that's how
> we've defined it so far.

How would a client handle this?  Does it need to provide a list of all media types it supports to the server, ala the HTTP Accept header?  That sounds overly complicated for a solution where text/plain is the primary use-case (see, e.g., JMAP preview).

More generally, I can provide this (real-world) example as my final argument as to the behavior at issue in this message.

Client:
  - images, videos, and URLs are commonly sent in messages
  - client wants to show small image preview if image is only content of message
  - client wants to show small image preview (or maybe a short GIF) if video is only content of message
  - client wants to show small image preview of website if a URL is only content of the message
  - server supports 3 PREVIEW extensions: IMG, VID, URL

The client, when listing (syncing) messages in the mailbox, will issue a FETCH PREVIEW for all of (IMG, VID, URL). These PREVIEW extensions describe mutually exclusive message data, so the client doesn't care which one is returned - it is identifying to the client what it supports display-wise.  For this client, it might make the decision whether to FETCH BODY text based on whether preview data is returned for a message.

This scenario is describing the UI behavior of every chat-like client out there: whether it be SMS/MMS, WhatsApp, iMessage, etc.  So it is extremely common, and generally reflects the experience a user expects when using messaging.  So a preview specification should theoretically be extensible to handle this kind of use-case.

To me it makes the most sense to allow this in a single FETCH call - one of the stated goals of the FETCH PREVIEW spec is to reduce roundtrips.  A "priority list" of algorithms is nothing more than a potential list of sequential FETCH calls the client might make.

michael