Re: [multipathtcp] Feedback on sha1/key exchange for MPTCP inter-op with FreeBSD

[Nigel Williams <[e-mail]>]

Hi Christoph,

Thanks for the response, it's clear where the misunderstanding is now.

On 16/10/12 17:32, Christoph Paasch wrote:
> Hello Nigel,
>
> On Tuesday 16 October 2012 15:14:07 Nigel Williams wrote:
>> (1) Endianess and the sha1 hashing of local and remote keys: I initially
>> had some trouble matching keys and idsns with the Linux implementation.
>> Having a look through the linux code (function mptcp_key_sha1(local_key)
>> in mptcp_ctrl.c) it looks like the output of sha_transform is converted
>> into big-endian, and the token and idsn are then assigned (as big-endian).
>>
>> Once the mptcp_key_sha1() function returns, 'idsn = ntohll(idsn) + 1;'
>> is called (returning the idsn to host order). The idsn is then typcast
>> as a u32 and this value is used for tcb->seq (which is the starting dseq
>> value of DSN mappings). All this was calculated from 'local_key', which
>> is stored locally in host order.
>>
>> The problem arises then that the local_key is put onto the wire in host
>> byte order, when we were expecting to receive it in network byte order.
>> Is there a reason why the key is not transmitted in network byte order?
>
> the key is not stored in host-byte order. When generating the random bits we
> simply store them in a u64 and send them on the wire. Thus, the keys are in
> network byte-order.
>
> The endianness conversion in mptcp_key_sha1 is part of the SHA-1 algorithm.
> SHA-1 does an endianness conversion on the input (done by sha_transform in
> lib/sha1.c).
> When returning the final output another conversion is done in sha1_final
> (crypto/sha1_generic.c), part of the generic crypto-library of the linux
> kernel.
>
> However, in MPTCP we don't use the crypto-library (thus, don't call
> sha1_final) because we want to spare some CPU cycles. So, we have to do the
> endianness conversion at the end of mptcp_key_sha1 by hand.
>

What caused the confusion is that the Linux key is considered to be in 
network order after it is generated. I don't recall the draft specifying 
what order the key is to be in when hashed, and it is apparent that it 
should.

I would argue in favour of hashing in host order for semantic reasons:
- It seems non-standard behaviour to not perform a network-to-host byte 
order conversion on arriving options.
- Maintains consistency with other stack variables which are kept in 
host byte order for simplicity of arithmetic (or other things)

>> Also, is it possible to do the conversion to host order of the idsn and
>> token within mptcp_key_sha1(), rather than on return? A quick glance at
>> the function does not make it immediately obvious that the values should
>> be converted into host order on return. Some comments in this function
>> would be useful (for instance the FreeBSD sha1 returns the hash in
>> network order, while Linux does not).
>
>  From what I see in
> http://www.leidinger.net/FreeBSD/dox/crypto/html/d2/d24/sha1_8c_source.html
>
> the sha-1 algorithm in FreeBSD is also endianness-agnostic. Meaning, if you
> call sha1_result at the end, he will also do an endianness conversion so that
> your output matches the endianness of the input.
>

Okay. Misunderstood what the Linux code was doing (we thought the 
FreeBSD code was doing something different to Linux, and this is not the 
case).

>> (2) Perhaps as a general comment/feedback, it might be worth detailing a
>> little more explicitly in the implementers guide the conversions
>> occurring in this process, as it is currently a little ambiguous and I
>> needed to dig through the linux code in order to make my implementation
>> compatible on the wire.
>
> I will try to add more comments in the implementation.
>
>> As far as I can tell, the process should be along the lines of:
>>
>> 1. generate 64-bit key (store in host order)
>
> In a certain sense, the endianness of the key doesn't matter. Because, no
> arithmetic operations are done on the key.

As the endianness of the key will change the hash output, it does matter.

cheers,
nigel