Re: [Multiformats] IETF 116 Dispatch Multiformats: Responses to Martin Dürst and Murray Kucherawy

["Martin J. Dürst" <[e-mail]>]

Hello Manu, others,

On 2023-04-11 01:47, Manu Sporny wrote:

[see below]

> Martin Dürst said:
>> For the "Hello World" example, can you express the charset?
> 
> The "hello world" example used during dispatch was expressed in
> Multibase, which is an ASCII format. All other Multiformats are binary
> formats, where some of the text-based formats express what charset to
> use. For example, the Multicodec entry for "JSON" says that the data
> is UTF-8 encoded).
> 
> There are no UTF-16 entries that I'm aware of, most likely because
> Multiformats are rarely used to express string values. The
> "plaintextv2" entry is missing a charset designation (which is likely
> a registration error). One presumption we could make is that the
> formats that carry the Multiformat value (such as CBOR, YAML, and
> JSON) already have clear rules around charset  encoding OR, if
> Multiformats are used as a byte header for a JSON or CBOR object, then
> the corresponding format has clear charset rules that should be used.
> 
> That said, your concern is not entirely addressed. There is nothing in
> the documentation around Multiformats that I have read to date that
> gives a clear and definitive answer to your question.
> 
>> These questions need to be answered, not sure what alternative might be used for that.
> 
> Your charset question could be a signal that a mini-WG might be a
> better option. When you asked your question, my internal dialogue was:
> "Oh, that's interesting... why has that not come up in the
> Multiformats community at all during the last 5+ years? And why is the
> ecosystem able to operate w/o answering that question?". It might be
> that I'm unaware of where that discussion happened, but I agree with
> you, we should have a definitive answer.

I have seen many, many communities over the years where questions such 
as "why has that not come up" and "why is the ecosystem able to operate" 
were quite appropriate. It's often that in a small(ish) community, not 
so many languages/OSes/... are involved to create problems, or that 
development mostly happens among people who speak English to each other 
(even if they may use other languages at home), and so data stays mostly 
ASCII anyway. Also, in recent years, the chance that everybody just 
picks UTF-8 is getting higher, because many languages and libraries 
expose UTF-8 to the outside (even if they may use UTF-16 internally in 
some cases).

Anyway, as you say, it's way better to nail these things down clearly.

Regards,   Martin.


> -- manu