Re: [apps-discuss] Scope of RFC3986 and successor - what is a URI?

[[e-mail] (Henry S. Thompson)]

Sam Ruby writes:

> As a concrete, current, and actionable topic, apparently it is
> desirable and common practice to include square brackets in the
> search/query component of "clean URLs":
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=1121826

So I want to take this (small) part of your message to try to
understand your implicit requirements process.

The bug report referenced there has a few key factual nuggets:

  1) 3986 [1] does not include '[' and ']' among the URL code points;

  2) The WHATWG URL document is tricky on this point as it stands
     today (2015-01-15): ']' and '[' are not allowed in queries, it is
     a parse error if they do, but the implicit error recovery step
     taken when they _do_ appear is to include them in the parsed URL
     without escaping them.

  3) IE8 and Chrome? do not %-encode '[' or ']' in the query string,
     whereas Firefox35 does %-encode;

On this basis you assert "apparently it is desirable and common
practice to include square brackets in ... 'clean URLS'".

My concern here is _not_ with the specifics of how square brackets are
_or_ should be handled, in browsers or in specs, but rather with how
we should make decisions about such questions.  What can we conclude
based on the facts listed above in this regard?  Insofar as the two
documents involved overlap, they agree -- a string with '[' and/or ']'
in what would otherwise be the query part of a URI/URL is not in fact
a URI/URL -- it fails to conform to the respective definitions
thereof.

The WHATWG document goes further, into a space which is evidently
out-of-scope for 3986, and defines an error recovery procedure for
such strings.  That procedure, it should be noted, _does_
percent-encode some other non-allowed characters when they appear in
the query part, including e.g. '<' and '>'.

Standardising error recovery is clearly a reasonable goal, once the
decision not to take a draconian approach is made.  But in the
(manifest) absence of agreement between existing implementations as to
what that error recovery should be, how should a choice be made?

We now return to your statement, quoted above, as it appears to
contain an answer to that question -- might makes right.  That is,
because Chrome and IE don't %-encode as part of error recovery, and
between them they have more market share than Firefox, Firefox loses,
and must stop doing it.  Obviously that's a caricature of an argument,
not an actual argument.  I'm confident you _have_ an actual argument
in mind, but I can't figure out what it might be.  Please enlighten
me.  Note that "because the WHATWG spec says so" is not sufficient,
because there must be a reason _why_ it says so.

ht

[1] https://tools.ietf.org/html/rfc3986
[2] https://url.spec.whatwg.org/

**Appendix**

The relevant bits of the WHATWG URL document are as follows:

      "A query must be zero or more *URL units*." [3]
      "The *URL units* are *URL code points* and percent-encoded bytes." [3]
      "The URL code points are ASCII alphanumeric, "!", ... [a list
       not including '[' or ']']" [3]

      "[if c is '[' or ']'], run these steps:
         1. If c is not a URL code point and not "%", *parse error*.
         2. ...
         3. Append c to buffer.

       [after processing all the characters into the buffer]
        ...
          3. For each byte in buffer run these subsubsteps:
             1. If byte is less than 0x21, greater than 0x7E, or is
             one of 0x22, 0x23, 0x3C, 0x3E, and 0x60, append byte,
             percent encoded, to url’s query.

          2. Otherwise, append a code point whose value is byte to
          url’s query." [4]

  The 'must' in the first quote above is a 2119 'must'.

[3] https://url.spec.whatwg.org/#url-writing
[4] https://url.spec.whatwg.org/#query-state
-- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: [phone], e-mail: [e-mail]
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]