Re: [apps-discuss] Fun with URLs and regex

[Matthew Kerwin <[e-mail]>]

On 29 January 2015 at 11:24, Roy T. Fielding <[e-mail]> wrote:

> On Jan 28, 2015, at 3:59 PM, Matthew Kerwin wrote:
> >
> > Whether or not I mention it comes back to the definition and intended
> > use-case of RFC 3986; if it defines an 'abstract' syntax - in the POO
> > sense - then there's no such thing as a universal parser (i.e. It's
> > impossible to parse a URI with an unknown scheme). If it defines a
> > low-level structure, then any URI can be parsed, but the individual
> > components can't be validated without deferring to scheme-specific
> > machinery.
> >
> > If the former and I don't include the fragment in 'file', it isn't
> > allowed. If the latter, I just leave a hole in the spec.
>
> It isn't that black and white.  The grammar for the scheme is what
> excludes a fragment.  That doesn't prevent the scheme docs from
> talking about fragments (in reference to RFC3986) and using them
> within examples.
> ​
> ​
> ​
> ​
>
>
> ​​
> That part of the URI spec was written specifically to address
> issues created by folks who thought they could redefine the meaning
> of fragments within individual schemes, or forbid them entirely,
> when in fact the meaning and use of fragments are independent of
> scheme.
> ​
> ​
> ​​
>

Poking around again, I just saw the line "Fragment identifier semantics are
independent of the URI scheme and thus cannot be redefined by scheme
specifications." So yes, I think I now understand where everyone is coming
from in the discussion, and I also think I understand RFC 3986 less than I
did before. Time for more reading...

RFC 3986 "defines a grammar that is a superset of all valid URIs, allowing
an implementation to parse the common components of a URI reference without
knowing the scheme-specific requirements of every possible identifier",
that's clear enough - it's not an abstract grammar. The twist that I'd
missed was that, of the resulting components, not all of them are inputs to
the individual schemes; the 'fragment' component feeds into the content
handler directly.

I'm still suffering a misalignment: RFC 3986 defines the whole generic URI
syntax as:

    URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

and my draft that references it essentially defines (or will soon define)
the whole file-URI syntax as:

    file-URI = subset-of-scheme ":" subset-of-hier-part

By leaving off the query part, I've either said that a URI with a query
part cannot be a 'file' URI, or that a URI that starts with "file:" and has
a query part is invalid. Potayto potahto.

I don't know what I've said by leaving off the fragment part. According to
RFC 3986 I'm not allowed to touch the it, but now I have a collected ABNF
for 'file' URIs that doesn't have fragments. I think my way forward is to
include the fragment part in the grammar, and then deflect it the way 3986,
7230, etc. do.  "The fragment's format and resolution is ... dependent on
the media type of a potentially retrieved representation, even though such
a retrieval is only performed if the URI is dereferenced. ... The [client]
MAY either assume a media type of "application/octet-stream" or examine the
data to determine its type." Or something to similar effect. Does that seem
right to you? How do other representation-retrieving schemes deal with it?

​​
>
> What makes you think that dereferenced files don't have a well-defined
> content type?  The client might not know what it is, but that doesn't
> mean the content type doesn't exist, and any decision to process the
> file is basically an assumption of some content type (and its rules
> for processing fragments).
>
>
Perhaps I s​hould have said "well known" instead of "well-defined," or that
the *means of determining* the content type is not well defined.


Back to Sam's library:

> Based on this discussion, I am gathering that the correct way to validate
a URI with a known scheme is as follows:
>
>   return new RegExp("^" + known[scheme] + "($|#" + fragment +
")").test(string)
>
> Anybody care to confirm or deny?

What with all the reading and thinking I've just done, I suppose that's
right. As long as all the known schemes don't have their own #fragment bits
(http_URI does).

Aesthetically it's strange way to denote an optional fragment, but it gets
there in the end.



-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/