Re: [Extra] Barry Leiba's Discuss on draft-ietf-extra-imap-fetch-preview-03: (with DISCUSS and COMMENT)

[Barry Leiba <[e-mail]>]

Excellent, Chris; thanks for the detailed response, and I'm happy with
that part of the DISCUSS.

The part I'm going to save for future reference is this one:

> I happen to consider the tagged/asynchronous part of the IMAP model to
> be an elegant design that is a failure in practice when it encounters
> the real world of implementations. The fact is the majority of clients
> and client APIs just get the model wrong -- they're so used to
> request/response that they try to shove a cache/update model into a
> request/response model with predictably bad results.

Thanks for that.

Barry

On Mon, Apr 8, 2019 at 3:38 PM Chris Newman <[e-mail]> wrote:
>
> On 7 Apr 2019, at 20:25, Barry Leiba via Datatracker wrote:
> > Barry Leiba has entered the following ballot position for
> > draft-ietf-extra-imap-fetch-preview-03: Discuss
> >
> > When responding, please keep the subject line intact and reply to all
> > email addresses included in the To and CC lines. (Feel free to cut
> > this
> > introductory paragraph, however.)
> >
> >
> > Please refer to
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_iesg_statement_discuss-2Dcriteria.html&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=MX001Ss_AeNyXvo6ikSivMny92vtZHwv1o5u46suxv0&s=E7J5zigfcri3VIAMhsM-C7rxlMvQsinhAupR0vFaE4Q&e=
> > for more information about IESG DISCUSS and COMMENT positions.
> >
> >
> > The document, along with other ballot positions, can be found here:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dietf-2Dextra-2Dimap-2Dfetch-2Dpreview_&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=MX001Ss_AeNyXvo6ikSivMny92vtZHwv1o5u46suxv0&s=JGZ_W2tQh-5CzY9IdDs4v_W8HvA2t830SFv9RiL8Snw&e=
> >
> >
> >
> > ----------------------------------------------------------------------
> > DISCUSS:
> > ----------------------------------------------------------------------
> >
> > — Section 3.1 —
> >
> > I don’t understand “the client’s priority decision”: what
> > decision is that?
> > And what’s the point of giving the server a list of algorithms here,
> > given that
> > they all have to be ones that are supported by the server?  Won’t
> > the server
> > always have to use the first one in the list?  If not, please add some
> > text
> > explaining what the server does.
> >
> > — Section 3.2 —
> >
> >    If the preview is not available, the server MUST return NIL as the
> >    PREVIEW response.  A NIL response indicates to the client that
> >    preview information MAY become available in a future PREVIEW FETCH
> >    request.  Note that this is semantically different than returning a
> >    zero-length string, which indicates an empty preview.
> >
> > I think the MUST here is hard to follow, because the text doesn’t
> > make a clear
> > enough distinction between “preview is not available” and “an
> > empty preview”.
> > Can you expand the text a bit to explain the distinction more clearly,
> > as this
> > is a protocol requirement?  Also, as I noted in response to Meral’s
> > Gen-ART
> > review it would be good to be clear how encrypted messages should be
> > handled in
> > this regard.
> >
> > — Section 4.1 —
> >
> >    The preview text MUST be treated as text/plain MIME data by the
> >    client.
> >
> > I think this requires a normative reference to RFC 2046.
> >
> > — Section 5.1 —
> >
> > The way you have LAZY working isn’t really consistent with the IMAP
> > protocol
> > model.  In that model, the client would not have to ask for the
> > preview twice,
> > one with LAZY and one without.  Instead, with LAZY, the server would
> > return
> > FETCH PREVIEW responses when it could — perhaps some in the first
> > set of FETCH
> > responses, and some, where the PREVIEW part was missing before, in
> > unsolicited
> > FETCH responses when the preview became available.  That way, the
> > server has
> > the responsibility of setting off a separate task to generate the
> > previews, and
> > to send them to the client when it has them (at which point it either
> > saves the
> > for future FETCHes or doesn’t).
>
> That alternative design would take control away from the client, make
> the client code more complex, and make the server code more complex. For
> clients using deployed request/response-style IMAP APIs, your proposal
> would violate the API model and thus be incredibly hard to implement
> without replacing a lot of the API. On the server side, even if I was
> willing to implement what you propose (and I'm not), I'd have to refuse
> to issue the untagged response until all PREVIEWs were generated to make
> sure request/response style client APIs could handle that server
> behavior -- at which point it's far less efficient than the proposed
> design where the client has control.
>
> I happen to consider the tagged/asynchronous part of the IMAP model to
> be an elegant design that is a failure in practice when it encounters
> the real world of implementations. The fact is the majority of clients
> and client APIs just get the model wrong -- they're so used to
> request/response that they try to shove a cache/update model into a
> request/response model with predictably bad results. In our 8.0 server
> release, we tried to push the envelope just a little bit by generating
> untagged EXISTS responses between commands (a behavior that's explicitly
> encouraged by the IMAP base spec) and that broke interoperability with
> more than one IMAP client in practice so we had to add an option to
> suppress that behavior to maintain interoperability with those
> (admittedly buggy) clients.
>
> > As it’s written here, the client has to open a separate IMAP session
> > with the
> > server and ask a second time for the previews it’s missing — a
> > separate session
> > to avoid blocking other action on the main session.  And if the server
> > has spun
> > off a task to preemptively generate them because the client asked once
> > (a good
> > practice, given the description here) it has to retain them for some
> > indefinite
> > period waiting for the client to ask again.
>
> As it turns out, that's exactly the implementation model followed by
> clients that generate previews manually without this extension. So with
> the new preview extension those clients just tweak the first command to
> add LAZY PREVIEW fetch so they possibly get early previews (improving
> the UI for a very small change). Then the client implementer can add an
> alternate code path to use the non-LAZY preview fetch variant instead of
> computing manually or the client implementer could ignore the non-LAZY
> preview feature and have fewer codepaths to test -- their choice (and
> the better choice might depend on whether the client is mobile or
> desktop -- something the server doesn't know).
>
> Since our server implemented PREVIEW before LAZY was added, I find the
> proposed LAZY an inconvenient but defensible addition to the protocol
> because it allows the client this additional implementation flexibility.
> But if the LAZY extension didn't give the client flexibility, then I'd
> consider the feature objectionable -- it would be significant
> unnecessary complexity without a defensible implementation benefit.
>
> > Why was this not done with the first mechanism?
>
> I can't speak to why the author wrote it that way, but as a server
> implementor, I find the way LAZY is presently specified to be perfectly
> acceptable and it has a straightforward implementation in my server.
> However, the proposed alternate design would be a royal PITA to
> implement on my server.
>
>                 - Chris
>
> > — Section 7 —
> >
> > As was mentioned in Ben’s review, either the ABNF for
> > “capability” is in error
> > (it should not include “preview-mod-ext”) or the description needs
> > to be
> > significantly beefed up.  I’m guessing that the intent is that
> > PREVIEW=
> > capabilities include both algorithms and modifiers, that PREVIEW=FUZZY
> > is
> > required, that the presence of any preview algorithm implies
> > PREVIEW=LAZY such
> > that the latter not only need not be specified, but is not permitted
> > to be.  So
> > we might have “PREVIEW=FUZZY PREVIEW=FURRY PREVIEW=SLEEPY”, which
> > would mean we
> > support the algorithms FUZZY and FURRY, and the modifiers LAZY and
> > SLEEPY.  Is
> > that correct?
> >
> > That seems somewhat obtuse to me, overloading the PREVIEW= capability
> > and
> > inviting confusion.
> >
> > — Section 8 —
> >
> > It seems like a bad idea to have to keep the IMAP Capabilities
> > registry in sync
> > with the two new registries: as it stands, when you add a new
> > algorithm you
> > have to add it to the Preview Algorithms registry, and also add a
> > corresponding
> > entry in the Capabilities registry... and similarly for a modifier, if
> > I have
> > that right above.
> >
> > Why not follow the model of AUTH= and RIGHTS=, and just reserve the
> > PREVIEW=
> > capability in the registry, allowing it to apply to entries from the
> > two new
> > registries?  That avoids inconsistencies in registrations if we later
> > add
> > algorithms or modifiers.
> >
> >
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> > — Section 3.1 —
> >
> > Nit: Please change “alternately” to “alternatively”.
> >
> >    These algorithms MUST be one
> >    of the algorithms identified as supported in the PREVIEW capability
> >    responses.
> >
> > There’s a number-agreement problem here.
> >
> > NEW
> >    All algorithms in the list MUST have been included in the
> >    list of algorithms identified as supported in the PREVIEW
> > capability
> >    responses.
> > END
> >
> > — Section 3.2 —
> >
> >    This relaxed requirement permits a
> >    server to offer previews as an option without requiring potentially
> >    burdensome storage and/or processing requirements to guarantee
> >    immutability for a use case that does not require this strictness.
> >
> > That’s sensible, but can you include some text giving an example of
> > a situation
> > where the preview might change?  Given that the messages themselves
> > are
> > immutable, why would applying the same algorithm to the same text give
> > different results?
> >
> > — Section 4.1 —
> >
> >    The server SHOULD limit the length of the preview text to 200
> > preview
> >    characters.  This length should provide sufficient data to
> > generally
> >    support both various languages (and their different average word
> >    lengths) and different client display size requirements.
> >
> >    The server MUST NOT output preview text longer than 256 preview
> >    characters.
> >
> > The text here should make it clear, because many implementers do not
> > understand
> > the difference, that these refer to *characters*, not *bytes*, and
> > that 200 or
> > 256 characters can possibly be much longer than 256 bytes.  I worry
> > that an
> > implementer might allocate a buffer of 256 bytes, thinking that’s
> > enough, and
> > have it overflowed.
> >
> >    The server SHOULD remove any formatting markup that exists in the
> >    original text.
> >
> > This is OK as it is, but perhaps a bit more specific than necessary.
> > I think
> > the sense is that the server is meant to do its best to render the
> > preview as
> > plain text, because that’s what the client will treat it as.  As
> > such, I would
> > fold this into the earlier paragraph that talks about no transfer
> > encoding, and
> > maybe say it something like this:
> >
> >    The generated string will be treated by the client as plain text,
> > so
> >    the server should do its best to provide a meaningful plain text
> > string.
> >    The generated string MUST NOT be content transfer encoded and MUST
> > be
> >    encoded in UTF-8 [RFC3629].  For purposes of this section, a
> > "preview
> >    character" is defined as a single UCS character encoded in UTF-8.
> > The
> >    server SHOULD also remove any formatting markup, and do what other
> >    processing might be useful in rendering the preview as plain text.
> >
> >
> > _______________________________________________
> > Extra mailing list
> > [e-mail]
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=MX001Ss_AeNyXvo6ikSivMny92vtZHwv1o5u46suxv0&s=N-3gUpkcA9xNcWGc9Rom-LAlShVKARcwVlk-0CUMjHc&e=