Re: [wasm] sqlite as rfc-spec'd web-interchange-format?

[kai zhu <[e-mail]>]

> nothing SQL-compatible is going to do the job.

we all acknowledge sqlite is not perfect, but do you have something that
could do the job better?  like keith said, its possible sqlite file-format
is the best choice that currently exists (for exchanging
relational-datasets over web).

> bincode, msgpack, protobuf, and of course, IETF's CBOR RFC7049.  Also
JSON.

i've had experience with protobuf, json, (plus json-schema and graphql).
as far as serializer/reviver ergonomics:
1.
json is idiot-proof for non-relational data.  json-schema is soso for
relational-data.
2.
protobuf and graphql are likely idiot-proof *if* you work at
google/facebook (and access to internal-tooling).  but my [frontend]
experience with their opensource serializers/revivers are disappointing and
error-prone.
3.
wasm-sqlite's serializers/revivers are idiot-proof for relational-data:
```javascript
// webassembly serializer:
FS.readFile("/sqlite-database.db") // return <serialized Uint8Array>

// webassembly reviver:
var dbRevived;
var ptr = stackAlloc(4);
FS.createDataFile("/", "sqlite-database.db", <serialized Uint8Array>, true,
true);
sqlite3_open("sqlite-database.db", ptr);
dbRevived = getValue(ptr, "i32"); // revived sqlite-database (in virtual fs)
...
sqlite3_exec(dbRevived, "SELECT * FROM my_table;", <callback>, 0, ptr);
...
```




On Sat, Nov 28, 2020 at 4:34 AM John C Klensin <[e-mail]> wrote:

>
>
> --On Saturday, November 28, 2020 01:14 -0500 Michael Richardson
> <[e-mail]> wrote:
>
> >
> > It's an interesting idea.
> >
> > There are quite a number of other serialization formats out
> > there including: bincode, msgpack, protobuf, and of course,
> > IETF's CBOR RFC7049.  Also JSON. For "opendata" we've would up
> > with CSV, which I really dislike. {So funny that the UK's data
> > problem with COVID rates was due to some data flow that went
> > CSV->XLS->database, fixed when they went CSV->XLSX->database,
> > when there was never a reason to use XLS* at all. So perhaps
> > this argues your case}
> >
> > The question you likely need to answer are:
> >
> > 1) do the sqlite people wish to turn change control over to
> > the IETF?
> >
> > 2) Given the Library of Congress statement, is there actually
> > a net benefit    to the world?  Maybe it's already moving to
> > defacto standard anyway.    Would the IETF process help?
> > Given the stability, would there be    any benefit for IANA?
> >
> > 2B) There may be NAFTA Article 10 ("Performance
> > Specification") reasons     for having an RFP'able
> > specification.
> >
> > 3) Are there enough implementers who are not sqlite.org who
> > would benefit?    Is the wasm-sqlite from sqlite.org (I don't
> > know).
> >
> > I looked through the specification, and turning it into an RFC
> > wouldn't be hard.  You may wish to look at the work in the
> > CELLAR WG, which is doing something similar for archival
> > formats for AV.
>
> I have not studied or used sqlite, but, stepping back from the
> details...
>
> Let me add something from a very different perspective -- maybe
> useful, maybe not.  If you are looking for a data serialization
> and interchange standpoint _and_ expect it to be able to
> accommodate the full range of statistical and scientific
> databases rather than only nicely rectangular, row-oriented,
> commercial databases and things that behave like them, nothing
> SQL-compatible is going to do the job.
>
> To save typing in a longer list, several of the papers in
> Rafanelli, M., Klensin, J.C., and Svensson, P., eds.,
> _Statistical and Scientific Database Management: Fourth
> International Working Conference on Statistical and Scientific
> Database Management, Rome, Italy, June 1988,  Proceedings_,
> Springer-Verlag Lecture Notes in Computer Science #339, 1989.
> (And, yes, for those who don't know, I had a life or three
> before getting sucked back, almost fully, into Internet work.)
>
>     best,
>       john
>
>
>
On Sat, Nov 28, 2020 at 4:34 AM John C Klensin <[e-mail]> wrote:

>
>
> --On Saturday, November 28, 2020 01:14 -0500 Michael Richardson
> <[e-mail]> wrote:
>
> >
> > It's an interesting idea.
> >
> > There are quite a number of other serialization formats out
> > there including: bincode, msgpack, protobuf, and of course,
> > IETF's CBOR RFC7049.  Also JSON. For "opendata" we've would up
> > with CSV, which I really dislike. {So funny that the UK's data
> > problem with COVID rates was due to some data flow that went
> > CSV->XLS->database, fixed when they went CSV->XLSX->database,
> > when there was never a reason to use XLS* at all. So perhaps
> > this argues your case}
> >
> > The question you likely need to answer are:
> >
> > 1) do the sqlite people wish to turn change control over to
> > the IETF?
> >
> > 2) Given the Library of Congress statement, is there actually
> > a net benefit    to the world?  Maybe it's already moving to
> > defacto standard anyway.    Would the IETF process help?
> > Given the stability, would there be    any benefit for IANA?
> >
> > 2B) There may be NAFTA Article 10 ("Performance
> > Specification") reasons     for having an RFP'able
> > specification.
> >
> > 3) Are there enough implementers who are not sqlite.org who
> > would benefit?    Is the wasm-sqlite from sqlite.org (I don't
> > know).
> >
> > I looked through the specification, and turning it into an RFC
> > wouldn't be hard.  You may wish to look at the work in the
> > CELLAR WG, which is doing something similar for archival
> > formats for AV.
>
> I have not studied or used sqlite, but, stepping back from the
> details...
>
> Let me add something from a very different perspective -- maybe
> useful, maybe not.  If you are looking for a data serialization
> and interchange standpoint _and_ expect it to be able to
> accommodate the full range of statistical and scientific
> databases rather than only nicely rectangular, row-oriented,
> commercial databases and things that behave like them, nothing
> SQL-compatible is going to do the job.
>
> To save typing in a longer list, several of the papers in
> Rafanelli, M., Klensin, J.C., and Svensson, P., eds.,
> _Statistical and Scientific Database Management: Fourth
> International Working Conference on Statistical and Scientific
> Database Management, Rome, Italy, June 1988,  Proceedings_,
> Springer-Verlag Lecture Notes in Computer Science #339, 1989.
> (And, yes, for those who don't know, I had a life or three
> before getting sucked back, almost fully, into Internet work.)
>
>     best,
>       john
>
>
>