Re: If not JSON, what then ?

[Mark Nottingham <[e-mail]>]

> On 2 Aug 2016, at 1:53 PM, Willy Tarreau <[e-mail]> wrote:
> 
> Hi Mark,
> 
> On Tue, Aug 02, 2016 at 01:33:39PM +0200, Mark Nottingham wrote:
>> 1) Using the first character of the field-value as a signal that the encoding
>> is in use is interesting. I was thinking of indicating it with a suffix on
>> the header field name (e.g., Date-J). Either is viable, but I don't think
>> it's a good idea to reuse existing header field names and rely on that signal
>> to differentiate the value type; that seems like it would cause a lot of
>> interop problems to me. Defining a new header field (whether it's Date-J or
>> Date2 or whatever) seems much safer to me.
> 
> I had the same feeling initially but I retracted. I fear that having two
> header fields will result in inconsistencies between the two (possibly
> intentional when that may be used to benefit an attacker). We'd rather
> avoid reproducing the Proxy-Connection vs Connection mess we've been seeing
> for a decade, where both were sent "just in case".

I know, I don't like it either. I'm just concerned that if we keep the name the same, it's much more likely it's going to not be properly converted, and that could enable attacks too.

Stepping back, I think we're talking about a set of rules something like this;

A. For a newly defined header field that explicitly uses the new format, send it in the new format
B. For existing header fields, if their expression in the new format is defined:
  1. If you have evidence that your peer can accept the new header format, send them in the new format
  2. Otherwise, send them in the original format.
C. All other fields are always sent in the original, HTTP/1 format.

I.e., having both versions of a single header's semantics the wire at the same time is an error.

This means that the format of those headers is effectively a hop-by-hop attribute; you might have a situation where a non-format-aware node forces the hops surrounding it back to the original format (for headers with two different ways to express those semantics).

This gives me pause. Converting from new to old and back to new is very likely to tickle a lot of bugs and cause a lot of interop problems. So, we could say that conversion only happens as a downgrade; i.e., if the next hop doesn't support the encoding, you can downgrade, but you never upgrade it again to the new encoding.

Presumably, the last "hop" might be inside the origin server, when it converts those header fields into the old format for backwards compatibility with existing applications that aren't aware of the new format.

Applications that *are* aware of the new format will still need to handle the original format, because there will be clients / hops generating it for the foreseeable future. 

This kind of seems like a mess to me, and leads me to think that the only time we should attempt this is during a major protocol revision (i.e., h3), and even then, with great trepidation.

If we get that far, deciding how to signal which headers are encoded seems more manageable :)


> However if we enumerate certain header fields that would deserve being
> encoded differently and find a way to group them, we may think about
> sending a composite, compact header field for transport/routing, another
> one for the entity where available information are grouped when relevant.
> Then maybe it could be decided that when one agent consumes such a field,
> before passing the message it must delete occurences of the other ones,
> and/or rebuild them from the composite one, in order to avoid inconsistency
> issues.
> 
> We have more or less this regarding Transfer-Encoding which voids
> Content-Length, and the Host header field which must always match the
> authority part of the URI if present.
> 
> These are just thoughts, maybe they are stupid.

Not stupid at all, but I am concerned about adding too much "magic"; if implementations are doing too much on your behalf, issues will arise (see above).

Cheers,



--
Mark Nottingham   https://www.mnot.net/