Re: [wasm] sqlite as rfc-spec'd web-interchange-format?

[Keith Moore <[e-mail]>]

On 11/26/20 7:08 PM, kai zhu wrote:
> i'm noob to mailing-list, and if you think there's more appropriate 
> place for subject, please let me know.
>
> motivation:
> as product-developer, i waste huge amount of time reinventing 
> error-prone serializers/revivers to exchange complex-datasets across 
> web.  standardizing sqlite's file-format would reduce 
> integration-headaches between me / teammates / 3rd-parties, since we 
> could more easily agree to use wasm-sqlite's standard 
> serializers/revivers (instead of everyone reinventing their own for 
> the web).
>
> why sqlite as web-interchange-format?
> 1.
> wasm-sqlite's serializers/revivers are opensource and accessible to 
> web-clients
> 2.
> The [U.S.] Library of Congress Recommended Formats Statement (RFS) 
> includes SQLite as a preferred format for datasets 
> <https://www.loc.gov/preservation/resources/rfs/data.html>. [1]
> 3.
> sqlite's last file-format (v4) remains stable and unchanged since 2006
> 4.
> all sqlite file-formats since 2004 (v1, v2, v3, v4) are intended to be 
> forward-compatible with future sqlite releases
> 5.
> sqlite is the "Most Widely Deployed and Used Database Engine" (and 
> perhaps second most widely deployed software library, after libz). [2]
>
> the spec for sqlite's file-format is here [3].  i'm no expert at 
> reading it, but am wondering how feasible to translate it to an 
> rfc-spec (minus the journaling part)?
>
> -kai

I've been badly burned by the sqlite library more times than I can 
count.  Unless it's been fixed fairly recently, it is lousy at handling 
concurrent queries, so for that reason alone promoting the file format 
might be a disservice.

But maybe it makes some sense as a database interchange format. Maybe.   
In my experience a lot gets lost in translation between sqlite tables 
and other databases, but maybe not so much that it's useless as a way to 
convey datasets. It's probably a useful format for use in information 
transfer and maybe even simple single-user queries, but beware of using 
the sqlite library to make any kind of services providing concurrent 
access to more than one process.

I'm also not sure whether IETF can attract enough of the right kind of 
expertise to do a good job at this.

But I do suspect there's a need for something like this to be 
standardized, by someone.

Keith