Re: [Cbor] Handling duplicate map keys

[Laurence Lundblade <[e-mail]>]

> On Nov 24, 2019, at 2:48 AM, Jeffrey Yasskin <[e-mail]> wrote:
> 
>> I think you make a good point that if a protocol is going to specify "take first", it can probably specify "error on duplicate key" for the same cost. That might imply some much simpler text:
>> 
>> "Protocols based on CBOR SHOULD fail with an error if a map contains a duplicate key, except that if the key isn't used at all, they MAY ignore it instead. Protocols that do not reject duplicate keys MUST (?) document why the cost of rejecting duplicates is too high and why accepting them will not lead to security vulnerabilities. An array might be a better choice for such protocols.”
> 
> I think I’d invert it and say that protocols that require duplicate detection for security reasons should describe that requirement in security considerations so the implementor gets a good solid hint that they need to worry about it.
> 
> There's an interesting difference of approach here. It's plausible to say that protocol designers should pick the secure design only when they realize their design has security implications, but I prefer to say that they should pick the secure design unless they think about it and realize the design doesn't have security implications. I think the second will get us noticeably more security in a world where designers don't have time to think about every aspect of every piece of their design.

I don’t know how many generic decoders exist that really can’t do or support duplicate detection. I also don’t know how many use cases there are that can’t afford duplicate detection. If there are few of both, then making no duplicate detection the exception that needs to be called out seems right.

I have been thinking about CWT and EAT as examples. Maybe EAT should say that duplicate detection should be required unless a profile of EAT says it is not, the secure choice. For the most part EAT will be processed on servers where there is clearly no resource issue. The only issue would be the implementation  environment (Java, PHP, Python…).

EATs might be decoded on constrained devices sometimes, but that is rare. In that case a profile can relax the requirement.

EATs are always signed so duplicates would only come from bad implementations. There are plenty of ways that am implementation can go wrong other than duplicating keys that are completely undetectable.

> 
> There is lot of text implying that duplicates are bad, but I think it would be worth being explicit.
> 
> You are an idiot if you design a protocol that considers duplicate input valid or necessary. Duplicate map keys are always an error in CBOR and will not work with many generic decoders.
>  
> I wouldn't say "idiot", just "wrong". The spec already says you "MUST NOT" do this, but I'm not opposed to making that statement more direct.

Pardon my wording. Certainly, I do not want that as normative text.

> 
> Thanks,
> Jeffrey