Re: [apps-discuss] draft-ietf-appsawg-xml-mediatypes vs. JSON and BOM and UTF-8

[Larry Masinter <[e-mail]>]

I said
> I don't really understand what a "XML-unaware MIME producer" is. If
> they're "unaware", why are they reading this spec?

But I think I'd missed this the first pass, I think I just don't like the -aware terminology here.
Please don't clarify.

There are three ways content can advertise its encoding:
  Charset parameter to media type
   Internal charset declaration
   Initial BOM (indicates utf16be, utf16le, utf-32, utf-8)

I think things get simpler if you treat the non-aware case just one of the exceptions to the "SHOULD" for transmitters.
That is, rather than dividing the world into "aware" and "not-aware", just make two categories:

* Those than can send UTF-8 without a BOM and with an internal charset declaration of UTF-8.
* Those that can't do so.

"producers of XML  SHOULD produce UTF-8, with no BOM, with no charset parameter, with an internal UTF-8 charset declaration", we make this the default; it is a new requirement. Those that can't are in some situation of deprecated legacy implementation or limited capabilities. They can't transcode into UTF-8, can't add or remove a BOM, or can't add or modify an internal charset declaration. Do the best you can, but try to make sure there's a charset parameter and that it matches the actual encoding or an internal charset declaration (because then processors will fail. 

Of course, there are more constraints on processors, because they need to deal with legacy, but this will give clear advice to senders.