Re: [I18ndir] Review volunteer needed (Fwd: [dispatch] WGLC of draft-ietf-dispatch-javascript-mjs-07)

[Asmus Freytag <[e-mail]>]

On 4/29/2020 9:17 PM, John C Klensin wrote:
> John and Asmus,
>
> It seem to me that we are straying into doing the design (or
> redesign) work that is the responsibility of the WG.  Explaining
> the problem to the WG and suggesting fixes would be entirely
> reasonable.   Trying to convince each other, IMO, a lot less so.
>
>     john
Good point - let's treat this discussion not as leading up to a
solution but to help us understand what requirements we'd

like to suggest to the WG.

A./

>
>
> --On Wednesday, April 29, 2020 19:37 -0700 Asmus Freytag
> <[e-mail]> wrote:
>
>> On 4/29/2020 6:45 PM, John Levine wrote:
>>> It looks like step 1 is saying that if the text starts with a
>>> BOM, you ignore the declared charset and sniff the BOM
>>> instead, which sounds to me like an ancient workaround that
>>> is perhaps no longer needed.
>> If data are preceded by ("start with")  a BOM, you do want to
>> strip it; you never want to keep it (and the chance that a
>> legitimate text starts with a BOM that has an actual function
>> in the text is perfectly negligible).
>>
>> If you have data that carries a UTF-16 bom it cannot be UTF-8,
>> no matter what the charset declaration says (FF can't be a
>> byte in UTF-8).
>>
>> Therefore, you always want to look for all three.
>>
>> If  it's UTF-8 you confirm you have the good character set
>> and remove it.
>>
>> If it's one of the UTF-16 ones, you switch the charset to
>> UTF-16 (or proper endianness) and remove it. (Or reject the
>> input?)
>>
>>> Given that they are deprecating all of the existing
>>> javascript media types and reviving text/javascript which
>>> 4329 declared obsolete, this might be a good time to say if
>>> you're going to use our lovely new (old) media type, declare
>>> the correct character set so consumers can believe it and
>>> stop doing byte sniffing kludges.
>> There are two issues here. One centers around the fact that
>> BOMs are "invisible". You can work hard at avoiding them, but
>> they may be added by some "helpful" tool.
>>
>> The other is that they serve as a useful consistency check
>> when present.
>>
>> You may write the spec to reject data with initial BOM, but
>> then you'd still need to check for them. You definitely don't
>> want to admit data without checking. Since you already need to
>> check for them, you are better off giving clear instructions
>> of how to make use of the fact that you've detected one.
>>
>> A./
>>
>