[Jmap] [Jmap for Sieve] Binary/blob vs. Inline

[Hans-Joerg Happel <[e-mail]>]

Hi,

as in today's JMAP WG call, and in prior threads on this list, there is 
some design decision to take on which minimal API calls are required to 
read/write a Sieve script.

The currently proposed procedure is non-atomic:

READ SCRIPT:
R1) obtain script "metadata" (including blobId)
R2) obtain actual script using blobId

WRITE SCRIPT:
W1) upload actual script as blob (obtaining blobId)
W2) set script "metadata" using SieveScript/set method


We see two issues with this approach:
1) It is not atomic, which may cause orphans to exist; resp. will 
require clients (or server) to do cleanup tasks (e.g., removing blob if 
second write call fails)
2) The whole approach seems to *assume the existence of a (global) blob 
store*, which *might not hold in general*


Point (2) has different implications for read and write:

=========================
READ
=========================
Assuming a JMAP backend might be heterogeneous (no single blob store; 
e.g: JMAP API encapsulating an email server and a separate Sieve 
server), the backend implementation might not "know" in step (R2) if a 
given blobId relates to an email attachment (to be obtained from mail 
store) or to a Sieve rule (to be obtained from rule store).

There might be two workarounds here:
* Encoding which blob store to look up in the blobId (e.g. using prefix: 
"sieve-123-123-123")
* Managing email + Sieve backend as different JMAP accounts; each with a 
logically separate blob store (as recommend by Neil in today's call)

=> So, this seems manageable


=========================
WRITE
=========================
When agreeing that JMAP for Sieve should be capable to wrap arbitrary 
existing Sieve systems (which may persist its scripts in files, LDAP, Db 
etc.), one might encounter backends which do not have a blob storage 
available. Let's just assume, the underlying Sieve storage is some 
database (with a fixed schema), which requires a single INSERT for 
script name + content and that would otherwise fail with an integrity 
constraint error.

In this situation, the current write process, as described above, would 
require the server-side JMAP for Sieve implementation to:
* Accept blob object written by client and *store temporarily* (e.g., 
write on disk?)
* Once the client calls "SieveScript/set", look up the blob file, read, 
merge + write to database

=> We'd argue that this process is both complex to implement plus it's 
not particularly elegant to create a need for storing a temporary file. 
It would therefore be good if it is possible to (alternatively?) submit 
the script inline in the "SieveScript/set" command instead of a blobid.

Best,
Hans-Joerg

ps.: A more general question of the "write blob first" approach is 
(beyond just uploading Sieve scripts) in how far quotas are respected. 
If blob storage can be used for all types of data, but quota (both total 
and max-element-size) may differ for email attachments, Sieve scripts or 
files - how is the initial blob storage supposed to "know" which quota 
to check against?

=> This seems to be addressed by a separate quota for unreferenced 
blobs, which includes size and item count [1].

This seems not entirely elegant, if clients can upload large chunks of 
data based on a high "blob quota", but only learn from a subsequent 
command, that the server won't accept the previously uploaded file due 
to further type-specific quotas. Such things could e.g., cause trouble 
for a brute-force migration client, which concurrently uploads large 
emails or files, and which (due to imperfect implementation) keeps 
submitting data into the blob store, even if email or files are actually 
already overquota.

Probably not a very strong concern, but one might alternatively think 
about a way in which users signal their later usage intent (e.g. "email 
attachment") at blob upload time, so that the server can reject early?

[1] https://jmap.io/spec-core.html#binary-data- "The server SHOULD use a 
separate quota for unreferenced blobs to the account’s usual quota."

-- 
audriga GmbH
Durlacher Allee 47
76131 Karlsruhe
Tel: [phone]
Fax: [phone]

[e-mail]
https://www.audriga.com

Handelsregister: Amtsgericht Mannheim - HRB 713034
Sitz der Gesellschaft: Karlsruhe
Geschäftsführer: Dr. Frank Dengler, Dr. Hans-Jörg Happel
USt-ID: DE 279724142