Re: [Extra] imap4rev2: RFC3516 BINARY

Brandon Long <blong@google.com> Fri, 09 November 2018 23:18 UTC

Return-Path: <blong@google.com>
X-Original-To: extra@ietfa.amsl.com
Delivered-To: extra@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C2391277CC for <extra@ietfa.amsl.com>; Fri, 9 Nov 2018 15:18:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.501
X-Spam-Level:
X-Spam-Status: No, score=-17.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 80hVU3EgaCbK for <extra@ietfa.amsl.com>; Fri, 9 Nov 2018 15:18:08 -0800 (PST)
Received: from mail-yw1-xc29.google.com (mail-yw1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AE7A41274D0 for <extra@ietf.org>; Fri, 9 Nov 2018 15:18:08 -0800 (PST)
Received: by mail-yw1-xc29.google.com with SMTP id v5-v6so2235706ywa.13 for <extra@ietf.org>; Fri, 09 Nov 2018 15:18:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=r3CMU0BbUwOAROiYIwB4hz664V2MoZngYT6I26A23mM=; b=DEDGdDeHRtCnjp9n6x+dFyhX/7wZqX0+vcjotQ19vNBMPHS6CdrP2f//tWzjjy5uzB 7azBvbxuiXvgPJ0/vUX89pgi8gRcHU0vpoxX6A+mLhD7YsrfnSp2JiPni7Dly3aAN8M/ WeAr+1b1zlu1f0dbkp/nXjjgMrq/33R6IhwC2bUpNthr+Lj7GALwRnxnJydf/i0kdsm1 N2P3NxQ8OMkJfItkvaNUIRaCLP3fD1FLwEUaonoMBqZT8wF4yHSWHBox2Za/Ybn8MgNF ifKU+vSkLXi5rb6PUEDDXIDDUo6N/zlFWW8ZXXPVeSkwSTycYiViby1VRp6GwM1t8/4a nSmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=r3CMU0BbUwOAROiYIwB4hz664V2MoZngYT6I26A23mM=; b=fvbINur0IFg1xvF3D8Oz+p1y8UWIc2NlGkoKsczuj9yQDT26SS2EI+XeNjQpNON+ua 3lZZ7o3E7q8317vWvP0yDYXBksP/ZDZexiWuwebn+6Uz3wCPtecC5yjN9Uhhozc8Z3xI 3PmjE5AK3OINjR/883EPW3ffH8py2CopeHxORlAfSantAhSX+LTLO+aG5W0W4Uzx33Ns jF+GALsh4IFhWtPU9sH5L351DVPvWCE6VRVEt4TE92Qy/Elbf70MxF8EPi1pOpCcBxnE 5nXlfO+zc/NP1OOrZVoYMQugLPki8tgDG7CSr+pUthdTPXSYoEDJycdDLC2bqqsZlEs1 8oDA==
X-Gm-Message-State: AGRZ1gIan9TsWu6Sc9qXdlUl6i+dwlZjjJTw5QIXZ2ZvymynDg2m90cy ofLmxnJXj0jx2TsKbQLqkE7cCxTZlv7NCTNyjC2M
X-Google-Smtp-Source: AJdET5cLI/EMESoNqbiWsZSjxXn4WANAj7NAACCrTqJvGzU7H2c9hL+RUT0PDGi304BdbdzOfIJTYqUSC3I9LCLOTUE=
X-Received: by 2002:a0d:cf85:: with SMTP id r127-v6mr10864536ywd.109.1541805486298; Fri, 09 Nov 2018 15:18:06 -0800 (PST)
MIME-Version: 1.0
References: <7a10e2be-5192-4955-8bd8-aa81a18a240b@sloti7d1t02>
In-Reply-To: <7a10e2be-5192-4955-8bd8-aa81a18a240b@sloti7d1t02>
From: Brandon Long <blong@google.com>
Date: Fri, 09 Nov 2018 15:17:53 -0800
Message-ID: <CABa8R6sCOUr75bh8p0HPDW4cvj9eKMdtjnMmRyXKhPqyjxK9wA@mail.gmail.com>
To: Bron Gondwana <brong@fastmailteam.com>
Cc: extra@ietf.org, aamelnikov@fastmail.fm
Content-Type: multipart/alternative; boundary="000000000000538a65057a43903a"
Archived-At: <https://mailarchive.ietf.org/arch/msg/extra/2gPx9cNeymv5xPspZoFcBDnSBeE>
Subject: Re: [Extra] imap4rev2: RFC3516 BINARY
X-BeenThere: extra@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email mailstore and eXtensions To Revise or Amend <extra.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/extra>, <mailto:extra-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/extra/>
List-Post: <mailto:extra@ietf.org>
List-Help: <mailto:extra-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/extra>, <mailto:extra-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Nov 2018 23:18:11 -0000

BINARYMIMEWe ran into some issues trying to implement BINARY, and also
didn't see a large number of clients which supported it (looking back, we
did this work mostly in 2014, so a while back).  My memory is a bit sketchy
of everything we ran into.

If I recall, we had some questions with the standard, ie what happens if
you request binary for non-leaf branches of the MIME tree (section-binary
implies that there's something special about what you can request, but it's
actually just the section-part).  Ie, if you ask for BINARY on a
message/rfc822, are we supposed to decode each individual part in that
multipart?  Again, we could have just responded NO with unknown-cte, I
guess.  There was also the question of what to do with data we'd munged, ie
we have heuristics for extracting base64 data when there are multiple
sections of it, or where a mailing list appended a non-encoded footer on
base64 data, etc, we probably would have just sent the data we'd extracted
instead of giving an unknown-cte response, for example.

We also had some issues with binary append which I don't recall, though
that you can just not support with some minor annoyance (why it didn't have
a separate CAPABILITY for APPEND instead of just allowing the server to say
NO, I dunno).  If I recall, we were worried about what version to store and
what to vend to non BINARY requests, and how that interoperated.  I guess
this wouldn't have mattered that much, we already have a "buggy" APPEND by
the standard, since the standard basically says you have to accept whatever
crap they APPEND, even if not an email message, and we have some base level
requirements for message formatting.

Overall, it was a part of our investigation into using binary more in
emali, ie cte binary and smtp (BDAT and BINARYMIME).  We implemented BDAT,
but our testing basically showed very few clients or servers that could
handle cte binary with embedded nulls.

Perhaps a cleaned up more strictly specified BINARY could be included,
otherwise I think you'll get a lot of unknown-cte responses from folks who
have to claim support but can't.
For one, its not clear that you can stream a base64 decode, at the least
you'd need to do two passes, one to verify correctness of the base64 data,
and the second to stream decode it.

Brandon

On Thu, Nov 8, 2018 at 3:34 AM Bron Gondwana <brong@fastmailteam.com> wrote:

> This is one of the open issues from imap4rev2 - which extensions should be
> part of the mandatory baseline.
>
> This one might need some clarification from Alexey - I'm not sure why it's
> not trivially obviously a good thing to include.  There was some stuff
> about "ability to fetch the decoded size" discussed, but I didn't capture
> good notes.
>
> Arguments for and against, or additional discussion, in response to this
> please.
>
> Cheers,
>
> Bron.
>
> --
>   Bron Gondwana, CEO, FastMail Pty Ltd
>   brong@fastmailteam.com
>
>
> _______________________________________________
> Extra mailing list
> Extra@ietf.org
> https://www.ietf.org/mailman/listinfo/extra
>