Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)

Michael Slusarz <michael.slusarz@open-xchange.com> Mon, 01 July 2019 07:29 UTC

Return-Path: <michael.slusarz@open-xchange.com>
X-Original-To: extra@ietfa.amsl.com
Delivered-To: extra@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6F7F812000E; Mon, 1 Jul 2019 00:29:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.3
X-Spam-Level:
X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=open-xchange.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id utim4DiWkJ-d; Mon, 1 Jul 2019 00:29:16 -0700 (PDT)
Received: from mx4.open-xchange.com (alcatraz.open-xchange.com [87.191.39.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5A89612001B; Mon, 1 Jul 2019 00:29:16 -0700 (PDT)
Received: from open-xchange.com (imap.open-xchange.com [10.20.30.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx4.open-xchange.com (Postfix) with ESMTPS id CB72F6A28C; Mon, 1 Jul 2019 09:29:11 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=open-xchange.com; s=201705; t=1561966151; bh=bhJ6Tzbe/bnWypagtWJla9XBlf18OY8lbOT5DZIY8s0=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From; b=KDO5fxzJLJ8SsyC0Z1tFc9Ozz9hLO+i1boucys4FGjGilaKHv60vx2/0Mm1RIny5J 06FAwaKW5Ged0QJEKDqMEVIGXYEHKb7ZydcM+zmolL0uLbs/yzL1XmijMc0XjIt8+f /OIklcEZR8vdQ813SuD0j6Q5F9IM429WyYLGj9BlEnOCrsfEfhQgW4gAyqwwctGgnR 3yJ9HcxN8xaHnfNOzvL9yajBjoTCqdHGaQcsIGZVazFTTPrIY+E3RqRHxgYcO3/1jZ YixhPMVEhLtu+LIqHdad41SmMmb49Y2j3Nn6IH5INC9sIKCPDl0i1da9yT8zqN3uGK DqQuiEGglDHkQ==
Received: from appsuite-gw2.open-xchange.com (appsuite-gw2.open-xchange.com [10.20.28.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by open-xchange.com (Postfix) with ESMTPSA id BEFA13C0024; Mon, 1 Jul 2019 09:29:11 +0200 (CEST)
Date: Mon, 1 Jul 2019 01:29:11 -0600 (MDT)
From: Michael Slusarz <michael.slusarz@open-xchange.com>
Reply-To: Michael Slusarz <michael.slusarz@open-xchange.com>
To: Barry Leiba <barryleiba@computer.org>
Cc: extra@ietf.org, Bron Gondwana <brong@fastmailteam.com>, The IESG <iesg@ietf.org>, draft-ietf-extra-imap-fetch-preview@ietf.org
Message-ID: <891167002.11504.1561966151796@appsuite-gw2.open-xchange.com>
In-Reply-To: <CALaySJK+vvD-piU53RDRhrzG6Ei0+nZnm6b_=KpqK=DBOA59YA@mail.gmail.com>
References: <155469393077.18315.15660535375707491655.idtracker@ietfa.amsl.com> <1478535427.18024.1554953503900@appsuite.open-xchange.com> <CALaySJKJxBTw6ptgDNvDjaXKDA4_ZFrb5b2gcUbSqsv1zwZSJw@mail.gmail.com> <1755746109.39421.1559403977713@appsuite-gw1.open-xchange.com> <CALaySJK+vvD-piU53RDRhrzG6Ei0+nZnm6b_=KpqK=DBOA59YA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Priority: 3
Importance: Medium
X-Mailer: Open-Xchange Mailer v7.10.2-Rev6
X-Originating-Client: open-xchange-appsuite
Autocrypt: addr=michael.slusarz@open-xchange.com; prefer-encrypt=mutual; keydata= mQENBFdRf+ABCACjsnzeuEJqUrZHnmyTL0r8JwN0YF6ZS/hgHYx83/dhz2LRgq0bkl5FSYPc6ZY7j7G NqvvPxR4Ri6xfevym91IhdJbQaiJ2B0kWAz+p4H/iXhgZwsYdjFN/c3+MPEjSazCPwASWCDHv2ueCay 0YO2dmw9TnR6rA6GiReQPGumgCZJ4xX8AUBUQftdJOrq/fl1xpYWrsFrTIfBml1x3Q4Mf3+ocH7ZT/u SST2IJTx4a9szQcnrsVHn/Fc2wp4P4FkW87sQFpLND80E5VAwxEdCAtQrhoocUfmh3LyyIncAOIRdw0 aR/1PSgwuj8A2c1W0DRYuzCNaGveugCT6GmbF7qbABEBAAG5AQ0EV1F/4AEIAJ98e4BRvKeJSzSqybD ZzOcphkUy+PlMrhkQkc2m36U01+4E9AtUz3XGNIJE+in9yYKqXbtHJCavOPcFhktVGFvSADKT311ZXq Cx/ibvrWI/DmoBCCIY9iwrutJX08nOoj53wpiNVOuR6vGFJIHO7TNosPcWWsw16LiZIwo9GiU2KseU/ xW0h26ouPbVqVpFX1Tgv+xaCBF8rj4kgoghnvTVG5aEgF+QExOLqx6BmarePKboTFVnMk6NYAwC5TtJ DfpWqa/5vQa8oVAex09elUrN+IQM/tcbMG+tAe5mhjJCke96tdEno29KYjs3Ecl9t1GMcsVAM1k8D8J DJzPCB/cAEQEAAYkBMQQYAQIAGwUCV1F/4AIbDAQLCQgHBhUKCQgLAgUJEswDAAAKCRAH1y6/pl54M6 obB/9AbXRu5SYhYrmTMFGDNqq0BKeoeS1n6cA2rvYRPmmwKSd9/sZG6815X8worSUjPb4r0P/9UoUUy P99BIN0aKc/7baCCV2/00fITrW0sS5Es2xuDuwcRAwwMJX09yMDeCu6M0Y2kn8QjKr1Pu87isoxliQz QDdsYJd9b/iSPFAW+sV7xYlkVcw4XoXYolQTUjNBuWbl6tV6dQN66m6RnCnB1uolFtbZENRiVcHOAdI DmYMjX3UL8cMtqpMyAXe0HTkb1BK2I5m4Kz0thK+beBBLyd6M4bK45zI6L3f5oDOty9o2jAnxlSVdUi ZfBSBypOekOX0bw2w/4XtUoS9emDRk
Archived-At: <https://mailarchive.ietf.org/arch/msg/extra/m45x-CE8Op0elAbImwpWB40uOwE>
Subject: Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)
X-BeenThere: extra@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email mailstore and eXtensions To Revise or Amend <extra.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/extra>, <mailto:extra-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/extra/>
List-Post: <mailto:extra@ietf.org>
List-Help: <mailto:extra-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/extra>, <mailto:extra-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 Jul 2019 07:29:19 -0000

Apologies on taking too long to respond.

I neglected to update the draft with the other changes discussed in previous messages; I will upload that now.

See below for additional responses.


> On June 2, 2019 11:33 AM Barry Leiba <barryleiba@computer.org> wrote:
> 
>  
> I think we're still in significant disagreement here; I'd like others
> from the working group to comment, and see if that can move us in one
> direction or the other.
> 
> > > > I'll start with providing a (future) real-world example, and why this
> > > > behavior exists in its present form.  Given a hypothetical future
> > > > algorithm "IMAGE" (generate image/jpeg preview if image data exists in
> > > > message), I issue this preview command:
> > > >
> > > > a FETCH 1 (PREVIEW (LAZY=IMAGE FUZZY))
> > > >
> > > > This is asking the server: provide me an image preview of the message,
> > > > but ONLY if 1) the message actually contains image information and 2)
> > > > that image information is already generated or can be generated very
> > > > quickly.  Otherwise, fall back to FUZZY generation.
> > >
> > > Hm.  That really seems like a contrived example.  How would the client
> > > software possibly know whether there's useful information in the
> > > image, or whether the information in the accompanying text is more or
> > > less important?  Suppose I have two messages:
> > >
> > > 1. text/plain that basically says nothing; image/jpeg that has a
> > > diagram that explains everything
> > >
> > > 2. text/plain that has the entire real content of the message;
> > > image/jpeg of the senders company logo
> > >
> > > It's pretty clear from that explanation that for message 1 I want some
> > > sort of description of the image and for message 2 I want a preview of
> > > the text.  But how would the client software know that, and,
> > > therefore, know what to ask for?
> >
> > Is that what a typical received message looks like?  To me, that looks like the edge case.
> 
> It's not the "typical" message, but I don't think it's an edge case
> either.  I do often get messages where the only significant content is
> in an image.  (I've tried to dissuade those responsible from sending
> things that way, but they persist.)  So, no, I wouldn't call it an
> "edge case" either.  And even if it were, that's not really the point;
> see below.

Agreed that this is not really the point of this discussion.  We aren't discussing how an image preview algorithm would work, since in practice it would likely need to be designed to be smart enough to not return image data for things like embedded company logos. (This is approaching "what is an attachment?" territory.)


> > Seems that a server could figure out (through experience) when image
> > preview data should be calculated or ignored.  Image size, MIME
> > structure information, MIME order, etc. could all be used to reach some
> > sort of 95% accuracy threshold.  Maybe that doesn't happen day 1 of
> > implementation, but a competitive, reactive server implementation will
> > work on this.
> >
> > The same issue exists for text previews, FWIW.  What about a text/plain
> > + text/html alternative message, where text/plain is "Please click on
> > this URL to view the text part", and the text/html part is empty text
> > (it's a single image link).  A server can be programmed to be smart
> > enough to ignore that part re: preview.
> >
> > As written, we allow a server to be given the latitude to figure these
> > kind of issues out.  Closing down that opportunity because it might be
> > a difficult logical problem isn't the right approach from my view.
> 
> I note that in your response you say "server" four times, at least
> once per paragraph, and "client" never.  But my point (see above) was,
> "But how would the client software know that, and, therefore, know
> what to ask for?"  These are things a *client* is requesting; how does
> the client know whether to request that the preview be preferentially
> done on the image or on the text?  I contend that it doesn't, and
> can't, and that, therefore, it's strange to expect it to ask, and
> strange to say that the server has to take the client's "priority
> decision" into account.

Client: "Please provide me the richest content you can - richest content is defined via the priority list I provided."

I don't see how a client providing a priority order of preview algorithms is any different than what the client would do if it had to download the message and parse it for preview data?

Also, how this is any different than issuing two consecutive FETCH PREVIEW commands?  If the first PREVIEW command returns the empty string, a client is just going to send another FETCH command using the next PREVIEW algorithm in the list.  Thus, the client is indicating its "priority decision" by the order it is sending its commands.  Why not do that in a single command?


> I'm still looking for a real use case where the *client* is the entity
> that's better suited to asking for any specific action with respect to
> a message preview.

See below.

 
> > > Apart from that, if we're going to do any sort of text-based preview
> > > of an image, I would think we'd just say that when FUZZY is applied to
> > > an image, that's what it does.
> >
> > FUZZY is explicitly defined as text only output though, since that's generally mapping
> > what clients already do.
> 
> Well, but saying, "That's the way it works now," isn't really the
> point.  We need to be looking at how we *want* this to work going
> forward, and I think that having the *server* decide what the best way
> is to give a preview of a message is the right answer.  And if that
> means that the server decides to make a preview from the text/plain
> part, the text/html part, the application/pdf part, or the image/jpeg
> part (perhaps using OCR on the image), then I think that's fine.
> FUZZY is explicitly defined as text only, only because that's how
> we've defined it so far.

How would a client handle this?  Does it need to provide a list of all media types it supports to the server, ala the HTTP Accept header?  That sounds overly complicated for a solution where text/plain is the primary use-case (see, e.g., JMAP preview).

More generally, I can provide this (real-world) example as my final argument as to the behavior at issue in this message.

Client:
  - images, videos, and URLs are commonly sent in messages
  - client wants to show small image preview if image is only content of message
  - client wants to show small image preview (or maybe a short GIF) if video is only content of message
  - client wants to show small image preview of website if a URL is only content of the message
  - server supports 3 PREVIEW extensions: IMG, VID, URL

The client, when listing (syncing) messages in the mailbox, will issue a FETCH PREVIEW for all of (IMG, VID, URL). These PREVIEW extensions describe mutually exclusive message data, so the client doesn't care which one is returned - it is identifying to the client what it supports display-wise.  For this client, it might make the decision whether to FETCH BODY text based on whether preview data is returned for a message.

This scenario is describing the UI behavior of every chat-like client out there: whether it be SMS/MMS, WhatsApp, iMessage, etc.  So it is extremely common, and generally reflects the experience a user expects when using messaging.  So a preview specification should theoretically be extensible to handle this kind of use-case.

To me it makes the most sense to allow this in a single FETCH call - one of the stated goals of the FETCH PREVIEW spec is to reduce roundtrips.  A "priority list" of algorithms is nothing more than a potential list of sequential FETCH calls the client might make.

michael