Re: Determining which fields are structured

[Mark Nottingham <[e-mail]>]

Hi Justin,

> On 18 Nov 2021, at 3:21 am, Justin Richer <[e-mail]> wrote:
> 
> The question at hand: how do you know if a particular field is supposed to be structured or not?

We made an explicit decision that you'd have to have specific knowledge of the header in some way; you can't recognise whether a field is structured just by looking at it, nor can you tell which top-level type it is (list, dictionary, or item), necessarily. 

> While working on an implementation for HTTP Signatures, I put in a bit of code that simply tries to parse any field as a Dictionary, List, or Item, and if it doesn’t throw an error, marks it as whatever kind of structured field worked. This seemed to work for the most part, but I quickly hit one case that surprised me:
> 
> 	Host: example.org
> 
> This field parsed as sf-dictionary, which I wasn’t expecting at all because it doesn’t look or feel like a dictionary item. However, after talking with Mark Nottingham, it turns out that this fits the ABNF just fine: it’s a valid single-key dictionary with one key of “example.org” and no value, which is interpreted as a boolean “True” value.

Aside: the ABNF is only illustrative, the algorithms are normative.

> So with that I’d like to re-assert my support for the “Retrofit of Structured Fields for HTTP” draft: https://www.ietf.org/archive/id/draft-nottingham-http-structure-retrofit-00.html
> 
> In particular, if the WG picks up this document, I would also like to see us add a column to the HTTP Field Registry for SF type, and register all of the existing ones that we know work as different fields (leaving unknown or undefined ones as blank, maybe? Or something like “unstructured” if we know it’s specifically not meant to be structured, like Host): https://www.iana.org/assignments/http-fields/http-fields.xhtml
> 
> This resource would help code like mine, as I’d be able to pull in that table and have some sense of what to expect when trying to parse a given field.
> 
> This would also help push the goal of having any new fields be built using structured field types — new field definitions would be required to fill out that column when they register the field name.

I think that's reasonable. I could see adding an annotation as to whether the type is "native" (i.e., the field is specified as a structured field), or retrofit (i.e., it's not the normative parsing algorithm, but you might have some luck using it).

Cheers,


--
Mark Nottingham   https://www.mnot.net/