Re: [apps-discuss] draft-ietf-appsawg-xml-mediatypes vs. JSON and BOM and UTF-8

["Martin J. Dürst" <[e-mail]>]

On 2014/01/09 10:44, Larry Masinter wrote:
> I said
>> I don't really understand what a "XML-unaware MIME producer" is. If
>> they're "unaware", why are they reading this spec?
>
> But I think I'd missed this the first pass, I think I just don't like the -aware terminology here.
> Please don't clarify.
>
> There are three ways content can advertise its encoding:
>    Charset parameter to media type
>     Internal charset declaration
>     Initial BOM (indicates utf16be, utf16le, utf-32, utf-8)
>
> I think things get simpler if you treat the non-aware case just one of the exceptions to the "SHOULD" for transmitters.
> That is, rather than dividing the world into "aware" and "not-aware", just make two categories:
>
> * Those than can send UTF-8 without a BOM and with an internal charset declaration of UTF-8.
> * Those that can't do so.
>
> "producers of XML  SHOULD produce UTF-8, with no BOM, with no charset parameter, with an internal UTF-8 charset declaration", we make this the default; it is a new requirement. Those that can't are in some situation of deprecated legacy implementation or limited capabilities. They can't transcode into UTF-8, can't add or remove a BOM, or can't add or modify an internal charset declaration. Do the best you can, but try to make sure there's a charset parameter and that it matches the actual encoding or an internal charset declaration (because then processors will fail.

Why would the default here be *with* an internal UTF-8 declaration? 
Documents without such a declaration are also well-formed/valid UTF-8 
XML documents (assuming that they are well-formed/valid otherwise).

Moving to an UTF-8 only world, not having to declare UTF-8 anywhere 
should be part of that world. It is not for HTML, unfortunately, but it 
is perfectly possible for XML.

Regards,   Martin.

> Of course, there are more constraints on processors, because they need to deal with legacy, but this will give clear advice to senders.