Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)

Michael Slusarz <michael.slusarz@open-xchange.com> Mon, 01 July 2019 07:29 UTC

Return-Path: <michael.slusarz@open-xchange.com>
X-Original-To: extra@ietfa.amsl.com
Delivered-To: extra@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55DD812001B for <extra@ietfa.amsl.com>; Mon, 1 Jul 2019 00:29:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.199
X-Spam-Level:
X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=open-xchange.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3zZ9vf_dig8X for <extra@ietfa.amsl.com>; Mon, 1 Jul 2019 00:29:38 -0700 (PDT)
Received: from mx4.open-xchange.com (alcatraz.open-xchange.com [87.191.39.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 22B8312001A for <extra@ietf.org>; Mon, 1 Jul 2019 00:29:38 -0700 (PDT)
Received: from open-xchange.com (imap.open-xchange.com [10.20.30.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx4.open-xchange.com (Postfix) with ESMTPS id 92C406A28C; Mon, 1 Jul 2019 09:29:36 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=open-xchange.com; s=201705; t=1561966176; bh=mKSHCd/sXbrQufmuWh//chbaA1yjlqxM2hEWFiT5Njk=; h=Date:From:Reply-To:To:In-Reply-To:References:Subject:From; b=0lgGCyHcbZyYy97kFLKx1Mf9qrwQMzZeW7qBV/rYiXf+uRGi4d6gfxowLjfoae9on gyeeh9l28vs49PZE97d157W4Mr0msKgre++H6Z9cfAwj1m5FkLzE1sgVY9f5NqitnL YpfnC+K9JOkjCLAAEK7oXYySmw9WTkYU/cNk1pRhGgAh9ythOdBSWvrZeKtcfB1IFM trePn1UuKumSOZy19LtptOqen00DG0vVaxt5XSABLW9Yg/h8ZhzFmx4fHTGUcDJJdN 82V5dtdX+/DEqnNjohpdxp968Uc1Nc+IepnnZoba4Vm7vKiEAujiOd/apkmkoJjRnS KG8QkL5DV8HHQ==
Received: from appsuite-gw2.open-xchange.com (appsuite-gw2.open-xchange.com [10.20.28.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by open-xchange.com (Postfix) with ESMTPSA id 869F43C0024; Mon, 1 Jul 2019 09:29:36 +0200 (CEST)
Date: Mon, 1 Jul 2019 01:29:35 -0600 (MDT)
From: Michael Slusarz <michael.slusarz@open-xchange.com>
Reply-To: Michael Slusarz <michael.slusarz@open-xchange.com>
To: Neil Jenkins <neilj@fastmailteam.com>, extra@ietf.org
Message-ID: <855477053.11506.1561966175692@appsuite-gw2.open-xchange.com>
In-Reply-To: <70781805-f45f-43aa-b71b-a1862a3610bb@beta.fastmail.com>
References: <155469393077.18315.15660535375707491655.idtracker@ietfa.amsl.com> <1478535427.18024.1554953503900@appsuite.open-xchange.com> <CALaySJKJxBTw6ptgDNvDjaXKDA4_ZFrb5b2gcUbSqsv1zwZSJw@mail.gmail.com> <1755746109.39421.1559403977713@appsuite-gw1.open-xchange.com> <CALaySJK+vvD-piU53RDRhrzG6Ei0+nZnm6b_=KpqK=DBOA59YA@mail.gmail.com> <70781805-f45f-43aa-b71b-a1862a3610bb@beta.fastmail.com>
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Priority: 3
Importance: Medium
X-Mailer: Open-Xchange Mailer v7.10.2-Rev6
X-Originating-Client: open-xchange-appsuite
Autocrypt: addr=michael.slusarz@open-xchange.com; prefer-encrypt=mutual; keydata= mQENBFdRf+ABCACjsnzeuEJqUrZHnmyTL0r8JwN0YF6ZS/hgHYx83/dhz2LRgq0bkl5FSYPc6ZY7j7G NqvvPxR4Ri6xfevym91IhdJbQaiJ2B0kWAz+p4H/iXhgZwsYdjFN/c3+MPEjSazCPwASWCDHv2ueCay 0YO2dmw9TnR6rA6GiReQPGumgCZJ4xX8AUBUQftdJOrq/fl1xpYWrsFrTIfBml1x3Q4Mf3+ocH7ZT/u SST2IJTx4a9szQcnrsVHn/Fc2wp4P4FkW87sQFpLND80E5VAwxEdCAtQrhoocUfmh3LyyIncAOIRdw0 aR/1PSgwuj8A2c1W0DRYuzCNaGveugCT6GmbF7qbABEBAAG5AQ0EV1F/4AEIAJ98e4BRvKeJSzSqybD ZzOcphkUy+PlMrhkQkc2m36U01+4E9AtUz3XGNIJE+in9yYKqXbtHJCavOPcFhktVGFvSADKT311ZXq Cx/ibvrWI/DmoBCCIY9iwrutJX08nOoj53wpiNVOuR6vGFJIHO7TNosPcWWsw16LiZIwo9GiU2KseU/ xW0h26ouPbVqVpFX1Tgv+xaCBF8rj4kgoghnvTVG5aEgF+QExOLqx6BmarePKboTFVnMk6NYAwC5TtJ DfpWqa/5vQa8oVAex09elUrN+IQM/tcbMG+tAe5mhjJCke96tdEno29KYjs3Ecl9t1GMcsVAM1k8D8J DJzPCB/cAEQEAAYkBMQQYAQIAGwUCV1F/4AIbDAQLCQgHBhUKCQgLAgUJEswDAAAKCRAH1y6/pl54M6 obB/9AbXRu5SYhYrmTMFGDNqq0BKeoeS1n6cA2rvYRPmmwKSd9/sZG6815X8worSUjPb4r0P/9UoUUy P99BIN0aKc/7baCCV2/00fITrW0sS5Es2xuDuwcRAwwMJX09yMDeCu6M0Y2kn8QjKr1Pu87isoxliQz QDdsYJd9b/iSPFAW+sV7xYlkVcw4XoXYolQTUjNBuWbl6tV6dQN66m6RnCnB1uolFtbZENRiVcHOAdI DmYMjX3UL8cMtqpMyAXe0HTkb1BK2I5m4Kz0thK+beBBLyd6M4bK45zI6L3f5oDOty9o2jAnxlSVdUi ZfBSBypOekOX0bw2w/4XtUoS9emDRk
Archived-At: <https://mailarchive.ietf.org/arch/msg/extra/QBlNCYM3VrU4Q2WizUGNJIShtN8>
Subject: Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)
X-BeenThere: extra@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email mailstore and eXtensions To Revise or Amend <extra.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/extra>, <mailto:extra-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/extra/>
List-Post: <mailto:extra@ietf.org>
List-Help: <mailto:extra-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/extra>, <mailto:extra-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 Jul 2019 07:29:41 -0000

Neil,
Thanks for your comments.  Responses below.

On June 2, 2019 6:27 PM Neil Jenkins <neilj@fastmailteam.com> wrote:


I must admit to having skimmed this discussion, but it seems to me that this extension is getting way over-complicated. The key here is simply to answer the question: what is this for? The only purpose of this is to provide a way for clients to get a snippet of (plain) text to shown in the mailbox listing without downloading whole body parts and expecting a fast response from the server (like fetching common headers). This is a common feature and worth making more efficient. Trying to do anything else is making things more complicated for close to zero benefit.
I agree that a plaintext preview, to mimic what exists in current client UIs, was a main driving factor for creating this draft.  I disagree that this is the only potential use-case.

I would contend:
  • No server is going to implement more than one algorithm. Like with fuzzy searching, we should not specify the algorithm in the spec. Servers should be free to implement the best solution they can, which is likely to change over time as new edge cases are found and new capabilities are available (for example, skipping salutations, OCRing an image-only body, skipping broken auto-generated plain text components and generating the preview from the text/html version instead, etc.). The server should be free to choose which parts it thinks it should use to generate the preview.
We are in agreement here.  This is FUZZY.  We have defined some output requirements and limitations, but that output is up to the server.

  • The client either accepts the server is going to give it a good preview or it will ignore the whole capability and generate its own, at the expense of being slower and more data hungry.
  • The output is always just plain text. This is what is shown in mailbox listings as the preview in Apple Mail, Gmail, FastMail, etc. If you think an image or text/html preview is useful, I disagree, but regardless that should be a different spec rather than overcomplicating one that solves an actual use case today. (To be clear, I think the spec already says this, but some of the comments in this thread seemed to indicate to me people thought you might want different typed outputs in the future).
I disagree with this logic, as it is essentially "let's standardize what already exists in clients."  I'd rather design a standard that fits the current use cases AND can be extended to future use cases.

A real world example - we are currently working on an initiative to push email as a chat medium (Shameless plug: https://www.coi-dev.org/" rel="nofollow">https://www.coi-dev.org/).  In every chat client I have ever used, sending images and/or video is an important part of conversations.  When an image/video appears in a chat stream, the full contents are not generally displayed - it is a small preview of the image (or a small preview of a frame of the video).  That small image that is surely a "preview" of that message.  Mixed media previews are a very common paradigm in other messaging systems - why should email be any different?

We shouldn't have to come back in a year and propose a new "FETCH IMAGEPREVIEW" standard to support this - we should instead be able to propose a simple, optional extension to the base PREVIEW behavior.

  • (Perhaps controversial but…) If the server offers the capability it should always have the data readily available, so the LAZY modifier is not necessary. I would expect servers that implement this to generally calculate and cache a preview on delivery. If it's going to take a while, the benefit to the client is much less and it might as well just calculate its own. That way the client only has two code paths (server-preview or client-preview) rather than three (server delayed preview as well).
We index on delivery, but there's plenty of reasons why preview may not be available at a later time.  More important, a preview is not a fundamental part of the message data - it is data created via transformation of that data.  The original message cannot be recreated, but previews can - a server operator should be allowed to choose different storage and/retention policies based on that fact.

A client doesn't have to use LAZY.  A server can quickly support LAZY by returning NIL for every message requested - this doesn't add any appreciable overhead to the client/server interaction, as the client would need to issue a separate FETCH PREVIEW call regardless of whether LAZY was supported or not.  If LAZY is not something a client/server operator wants to deal with the complexities of, they can be avoided at almost zero development cost.

In the JMAP Mail spec, we simply specify the following for the equivalent property:

preview String
A plain text fragment of the message body. This is intended to be shown as a preview line on a mailbox listing, and may be truncated when shown. The server may choose which part of the message to include in the preview; skipping quoted sections and salutations and collapsing white-space can result in a more useful preview.

This MUST NOT be more than 256 characters in length.
PREVIEW=FUZZY and JMAP preview can be serviced by the same data (Bron suggested changes to the preview spec in the past to ensure this), so I think we are good here.

michael