Re: [Cbor] MIME tag 257 vs 36

[Laurence Lundblade <[e-mail]>]

Hi Carsten,

Thanks for the detailed explanation.

Is this a good summary of the upshot?

Text lines in Internet protocols (on the wire) are delimited by either a CRLF or just an LF. Officially many protocols specify CRLF, but implementations often work with either. CBOR type 3 text can be either line ending, even a mixture of both.

Operating systems usually have a line end convention. Windows uses CRLF. Linux and MacOS use LF. Some applications on a given OS may work with either and some may prefer the OS’s line ending convention.

The majority of use cases and CBOR protocols using type 3 text will work with either line ending. However, some use cases or protocols may not work with either in which case translation to and/or from the local line end convention, typically that of the OS, is necessary.

Are you proposing that CDDL be able to specify a line end convention for type 3 items?

LL



> On Sep 24, 2020, at 10:59 PM, Carsten Bormann <[e-mail]> wrote:
> 
> Hi Laurence,
> 
> On 2020-09-25, at 07:24, Laurence Lundblade <[e-mail]> wrote:
>> 
>> I did some more reading:
>> - CRLF is required for MIME headers, q-p, b64 CTEs (RFC 2045)
>> - CRLF is required for text/xxx types (RFC 2046)
>> - Unicode Format for Network Interchange requires CRLF (RFC 5198)
>> - HTTP headers require CRLF
> 
> This is one of the big areas where the letter of the standards and the reality can differ a lot.  Newer versions of the standards tend to contain a larger dose of reality; e.g., RFC 7230:
> 
>   Although the line terminator for the start-line and header fields is
>   the sequence CRLF, a recipient MAY recognize a single LF as a line
>   terminator and ignore any preceding CR.
> 
> RFC 2616 was somewhat more weasely:
> 
>   RFC 2046 requires that content with a type of "text" represent
>   line breaks as CRLF and forbids the use of CR or LF outside of line
>   break sequences. HTTP allows CRLF, bare CR, and bare LF to indicate a
>   line break within text content when a message is transmitted over
>   HTTP.
> 
>   Where it is possible, a proxy or gateway from HTTP to a strict MIME
>   environment SHOULD translate all line breaks within the text media
>   types described in section 3.7.1 of this document to the RFC 2049
>   canonical form of CRLF. 
> 
> Since HTTP is what happens for most interchange, expect the strict CRLF requirement of RFC 2046 to be ignored.
> 
>> - Git stores LF and translates to local line ending
> 
> Git is rooted in reality.
> (It gained CRLF support only after some gyrations.)
> 
>> These text formats choose a canonical line ending for on-the-wire messages so they can know to correctly translate to the local text format. These days it is just CRLF for Windows and LF for Linux/MacOS,
> 
> Right.
> 
>> it used to be CR for MacOS and in theory any local representation could be used including line lengths or a database of records.
> 
> I wish people would stop talking about CR line ends; these used to exist, but they no longer do in reality (outside very weird closed environments such as certain PDF files).  RFC 7230 reflects what the reality is, today, which is no longer very tolerant of pure-CR line endings (it’s been 20 years now since those became obsolete).
> 
>> It seems clear that tag 257 and 36 do require CRLF per the MIME standard. That’s all fine and there’s nothing more to do other than reference the MIME standard.
> 
> That’s what they say.
> Any implementation that doesn’t accept LF line endings is pretty much broken for the real world.
> 
>> You all may have though about this more than me, but it also seems there is room for the definition of some tags for major type 3 used with line-oriented text since major type 3 is silent on the line ending convention (as it should be). Maybe:
>> 
>>  Tag xx for IETF CRLF line endings (means translate CRLF to local; perhaps error on CR)
>>  Tag yy fo MNU (means translate LF or CRLF to local; perhaps error on CR)
> 
> It may not be necessary to tag something for what it already is.
> But it may be useful to be able to say in CDDL what a text string should look like  ; that may be something that could be added to the MNU specification or to a companion CDDL extension.
> 
>> CBOR protocols that use line-oriented data can specify which on-the-wire line ending they use, so the tags are not absolutely required, but they may be useful. They may also slightly help CBOR protocol designers using line-oriented text realize they need to say what the line ending is.
> 
> In reality, the line ending on the Internet is [CR] LF.  There is no other place CR can be used outside playing teleprinter (RFC 5198), so most software can simply treat CR as pure noise, which it typically does.
> 
> Grüße, Carsten
> 
> _______________________________________________
> CBOR mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/cbor