Re: [apps-discuss] Scope of RFC3986 and successor - what is a URI?

[<[e-mail]>]

>>Sam Ruby wrote:

>>
>>An example of something that is not a conforming URL, but is both a valid URI and a valid IRI:
>>
>>  http://257.257.257.257/
>


>Dave Cridland wrote:


>OK... I don't think that's valid, but it's probably valid according to the ABNF only (ignoring the prose). I think >specifying that in the formal syntax would be uncontentious - in fact, draft-ietf-iri-3987bis-13 already makes >this change.


It is not even valid based on the ABNF, due to a fairly creative definition of dec-octet.


https://tools.ietf.org/html/rfc3986#section-3.2.2


Darrel






From: Dave Cridland
Sent: ‎Thursday‎, ‎January‎ ‎15‎, ‎2015 ‎5‎:‎09‎ ‎PM
To: Sam Ruby
Cc: IETF Apps Discuss







On 15 January 2015 at 18:27, Sam Ruby <[e-mail]> wrote:

Good question.  Please forgive the lengthy answer.






Lengthy is fine.

 

Lets posit the existence of a W_URL parser.  Given a string, it produces an object.  One of the operations that object performs is "stringification" (an IDL term that I don't particularly care for which essentially means serialization to a string form).

 - - -

If you look at the sequence of operations: parse followed by stringification, you get a string transformation.

This can be an identity transform.  If you want, try the following in Firefox or Chrome's console:

  new URL('http://example.com/').href == 'http://example.com/'

The result will be true.

The result can be some transformation.  For example:

  new URL('HTTP://EXAMPLE.COM/PATH').href == 'http://example.com/PATH'

Finally, the result can be to give up.

  new URL('http://example.com:port/')

Both Firefox and Chrome throw an exception.

 - - -

Now if you take a look at that transformation, we can discuss the what should be desirable characteristics of the set of outputs.

The first characteristic that I would like to suggest is that the outputs round trip, and by that I mean that if you feed that output back into the transformation, you get the same result back.






OK; you're saying that for any URL string S, new URL(new URL(S).href).href == new URL(S).href




Your tests seem to imply that for any URL string S, there is one and only one correct value of (new URL(S).href); is that also the case?




To put it another way, your tests imply a canonical form for a URI, and that seems much more useful.

 

The second characteristic is that it actually works.  This isn't a precisely specified criteria, but it means that if you attempt to use the result it doesn't break anything.  One example of this would be placing the string 'http://ietf.org/' into the href attribute of an HTML <a> element, rendering it on the screen, and having somebody click on the link.  If this goes to the right place, then all is good.  The definition of "works" for data: W-URLs, and for javascript: W-URLs are different, but that's not an obstacle at this level of discussion.






I can readily appreciate the sentiment here; like many things, it's hard to specify. But what you're saying is that for any URL string S, where (new URL(S).href) does not produce an error, the result must be a semantically valid URL string? (i'm being delightfully wooly about "semantically valid").

 

 - - -

At this point, I'm troubled.  Despite the set of desirable characteristics I have specified, I have a well defined set which is neither a superset of I-URI's nor I-IRI's, nor a proper subset of either.

An example of something that is not a conforming URL, but is both a valid URI and a valid IRI:

  http://257.257.257.257/






OK... I don't think that's valid, but it's probably valid according to the ABNF only (ignoring the prose). I think specifying that in the formal syntax would be uncontentious - in fact, draft-ietf-iri-3987bis-13 already makes this change.




Similarly, http://127.0.0.1:65536/ is also not valid, but because the concept of port is not tied to a specific transport protocol, and might not be restricted to a 16-bit range, that's a harder one to capture in pure ABNF - that is, the permissable values of port are arguably scheme specific.




Both these are issues of what I called semantic validity above. Others probably exist.

 

Here is something that round-trips a URL parse/stringify transformation, and actually works, despite being an invalid URI and invalid IRI:

  https://www.google.com/?q=[z]






My gut feeling is that allowing this to be valid is a possible change, as I've said. I appreciate this is at odds with RFC 3986, but if URL parsers do generally ignore the error that's fine.

 

To further compound this, RFC 3986 claims that the following strings are to be treated as equivalent (and perhaps even preferred as they are deemed to be 'valid'):

  https://www.google.com/?q=%5Bz%5D
  https://www.google.com/?q=%5bz%5d

Indeed in many, and generally most, situations, those are considered equivalent.  But this is not necessarily so.  To it's credit RFC 3986 does describe a Comparison Ladder.  Unfortunately it does so using terms like "false positives" and "false negatives" when in fact, they aren't false at all.






I think that this would disappear, largely, with a canonical form. RFC 3986 currently says that pct-encoded should use upper case - that's a lower case "should" - and if we take this to be a canonical form, then the latter canonicalizes to the former, and we win. Of course, if [] is made legal in the query string it'd canonicalize to the previous example.




Comparison of URIs then become a case of comparing the canonicalized versions of both.




This is problematic in the case of ports - since http://www.google.com:80/ and http://www.google.com/ both refer to the same thing, but to have a single canonical form for both would require the parser to be aware of the default port for the scheme. But maybe unknown schemes cannot be canonicalized, or maybe we have two canonical forms. (Scheme-aware and scheme-agnostic; obviously you'd specify which you want and where).

 

 - - -

So much for a deep dive.  Returning to the surface: I'd like to simultaneously loosen up the definition of validity for I-URI's in some places while tightening it up in others.  This is a work in progress, so which I can't say exactly where just yet in each case, I can say that now would be a good time for interested parties to participate.


ht


- Sam Ruby


[1] https://url.spec.whatwg.org/, as of 2015-01-15




 _______________________________________________
apps-discuss mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/apps-discuss