Re: [apps-discuss] Review of draft-ietf-appsawg-file-scheme

[Matthew Kerwin <[e-mail]>]

Hi Dave,

Thanks for this review; it's nice and hefty. I'll work my way through it,
commenting inline where relevant, and will endeavour to incorporate your
suggestions into my working copy.


On 13 April 2016 at 05:29, Dave Crocker <[e-mail]> wrote:

>
>
> Review of: The file URI Scheme
> I-D: draft-ietf-appsawg-file-scheme-05
>
>      (I see that a -06 version was released today.  Based on a quick
>      scan of differences from the -05 version, I do not see their
>      affecting the review.)
>
>
> Reviewer:  D. Crocker
> Date:      12 April 2016
>
> This review was performed in my role as document shepherd.
>
> The review was originally written last November but unfortunately I lost
> track of it and apologize for the delay.
>
> I should also comment that I've just consulted a number of others who have
> been involved in discussions about the draft and hope that they will
> response to this review, on the mailing list.
>
>
>
> Summary:
>
> The specification seeks to detail the file: URI scheme, which "identifies
> a file on a particular file system", and replaces RFC 1738.
>
> The document has been housed within the Apps Area WG, but has received
> little substantive comment -- somewhere in the range of 5-8 people, with
> small bursts of activity. Although at least some have tended to be
> supportive, there have been notable exceptions. Worse, significant issues
> that were raised were not obviously resolved on the list.  (I looked at
> postings by others, immediately after the author's postings, to see whether
> there were statements of agreement that an issue had been resolved, and did
> not generally see them, concerning those substantive issues.)
>
>
> The draft began as an individual effort, in June 2013, went through many
> revisions, and was adopted by AppsAWG in January 2015, and has had a number
> of revisions.  I note a 9/26/2014 (pre-AppsAWG) comment from Daniel
> Stenberg:
>
>      "I would rather have a new spec straighten up and tighten the
>      language somewhat so that we can get a stricter interpretation of
>      how a file:// is supposed to work."
>
> which, in spite of the many document revisions, unfortunately matches my
> own assessment of the current draft.
>
>
This is a surprise, since this comment from Daniel was what triggered the
complete restructure of the draft into its current form (with a relatively
short normative body, and a bunch of informational stuff in appendices.)



> I also note a 9/15/2015 (post-AppsAWG adoption) comment from Mark
> Nottingham, on behalf of the W3C TAG:
>
>      Regarding the use of `file://` URIs on the web, there are a few
>      issues that need to be resolved for interoperability:
>
>      1. How does `file://` fit into the web's origin model?
>
>      2. How does retrieval of `file://` URIs fit into
>         [Fetch](https://fetch.spec.whatwg.org/) and/or the [URL
>         standard](https://url.spec.whatwg.org)?
>
>      This document does not address any of these issues, so we
>      encourage the APPSAWG to consider addressing these.
>
>
After some discussions with W3 and WHAT folk, I did include mention of
'origin' in the draft.

When I received the comments, fetch was very protocol-oriented, and 'file:'
doesn't belong to any particular protocol, so there was no obvious fit
there; and the draft as it was designed was a close match for the WHAT URL
standard as it was at the time (and from what I can see, still is now) so I
don't know what more need be said. If someone could tell me what sort of
thing they think I should be saying, then I could probably say it.



>
> One point of process discussion about the draft was to distinguish between
> documenting current practice, versus specifying enhancements. Another was
> to distinguish between use of the construct locally (within a user's own
> system) versus globally, across the public Web.  Again, neither matter
> appears to be resolved clearly within the draft, and I can't tell tell
> clearly which goals the draft targets, even with the Abstract text:
>
>      "It attempts to define a common core which is intended to
>      interoperate across the broad spectrum of existing
>      implementations, while at the same time documenting other current
>      practices."
>
> These two goals entail a classic choice in development of a specification
> that deals with existing practice.  Clarity between the two goals and
> careful precision about which is being served, is essential.
>
>
Yes, this could really do with rewriting and clarification. More on this
below, in your inline breakdown.



>
> A file: construct should be of significant utility to the Internet
> community.  So it warrants careful community review and extensive
> indication of active support.  That is, there ought to be a basis for
> assessing the likelihood of implementation and use.  As of now, this is not
> possible.
>
>
Technically it's already implemented in a bunch of places, not always
entirely interoperably. Most of the non-interoperable parts are in the edge
cases, but the "common core" is pretty universal. Since there's no
(non-obsolete) spec that defines it, what I want to achieve with this draft
-- if nothing else -- is to put that "common core" in a (non-obsolete) spec.

If that just meant de-obsoleting RFC 1738 that would almost be enough
(although an update might be in order.) However the discussion that started
with that question has lead us to where we are now, with this draft.

For what it's worth, at least one potential implementation is likely: the
Ruby standard library OpenURI module, which was the original reason I
started pursuing this in the first place (
https://bugs.ruby-lang.org/issues/8544)



>
> In technical terms, document seems to suffer some confusion about its role
> as a format specification, versus as a protocol specification.  I believe
> this issue is basic and important.  It needs to be resolved.
>
>
I don't disagree. Guidance on how to clarify this would be appreciated. How
many URI specs don't also define a protocol? If this is the only one, then
I probably don't know how to set the precedent. (Some of this is sorted
below, with your suggestions.)


For a specification involving such a potentially and presumably important
> capability, I think significant community support should be required...
> unless the spec is to be offered as Experimental, which is the most I'm
> inclined to recommend at this point...
>
>
To be honest, if that's what you think is most suitable and if we can't
improve it, I don't have a problem with Experimental. We're fighting
against decades of stagnation and ennui; if an Experimental spec can kick
some life into it and stir up the muck that's a good start. If something
resolves out of it, then there's nothing wrong with redoing it "properly"
in future, with more engagement and recent experience to call on.

But to reiterate, it's already implemented pretty widely so I don't think
it can count as an "experimental scheme" -- rather, this would be an
experiment in restandardising a diverged and stagnant scheme.



>
>
> Details:
>
>
> Applications Area Working Group                                M. Kerwin
>> Internet-Draft                                                       QUT
>> Obsoletes: 1738 (if approved)                           December 1, 2015
>>
>
> I believe it merely updates it.  It essentially replaces only Section
> 3.10 of that RFC.
>
>
Yeah, just like 4248 and 4266 did. I'll follow your guidance here. (If I
update an obsolete spec, do I inherit that obsoletion?)



> Intended status: Standards Track
>> Expires: June 3, 2016
>>
>>
>>                           The file URI Scheme
>>                    draft-ietf-appsawg-file-scheme-05
>>
>> Abstract
>>
>>    This document specifies the "file" Uniform Resource Identifier (URI)
>>    scheme, obsoleting the definition in RFC 1738.
>>
>>    It attempts to define a common core which is intended to interoperate
>>
>
> attempts to define -> defines
>
>
ACK


also: common core of what? I think the answer is a common core of object
> storage naming convention, but whatever is correct, it should be made
> explicit here.
>
>
A common core of what current file URI implementations support and do. At
the very least, a common core of syntax.



>
>
>    across the broad spectrum of existing implementations, while at the
>>    same time documenting other current practices.
>>
>
> implementations of ... URI-based file system accessing mechanisms?
>
>
Um, yes, I guess that's a way of putting it. Implementations of things that
use file URIs. Should I not write that is "implementations of this scheme"
or just "implementations"?



>
>> ​<snip>
>>
>>
>>
>> 1.  Introduction
>>
>>    A file URI identifies a file on a particular file system.  It can be
>>    used in discussions about the file, and if other conditions are met
>>    it can be dereferenced to directly access the file.
>>
>
> Since this is a specification, and it has intended for wide use, there
> should be some sort of basic definition of what a file system is.
> Nothing fancy. And by way of priming the pump:
>
>      ...on a particular file system, which is an object stored in a
> structured naming-and-accessing environment on a host. The URI can be
> used...
>
>
Hmm, ok. Are parenthetical definitions like this alright?

> A file URI identified an object (a "file") stored in a structured
> object naming and accessing environment on a host (a "file system.")
> The URI can be used ...



>
>
>>    The file URI scheme is not coupled with a specific protocol, nor with
>>    a specific media type.  See Section 3 for a discussion of operations
>>    that can be performed on a file URI.
>>
>
> I think these defined operations are not really performed 'on' the
> file URI, but rather are used to create or apply the URI.
>
>
Translating to a local file path is one you perform on the URI itself. How
about "...operations that can be performed on a file URI or the object it
identifies"? Or just "...on the object identified by a file URI" if you
don't like the transformation.



> Also:
>
>    media type. -> media type [rfc6838].
>
>
Sure. Is that an informative reference, or normative? We're mentioning it
as something that doesn't apply here, so I'm not sure.



>
>    This document defines a syntax that is compatible with most extant
>>    implementations, while attempting to push towards a stricter subset
>>    of "ideal" constructs.  In many cases it simultaneously acknowledges
>>    and deprecates some less common or outdated constructs.
>>
>
> *** ideal?
>
> How does it 'acknowledge' such constructs?
>
>
I will replace this whole paragraph, it's no longer correct. How about:

> This document specifies a syntax that is compatible with most extant
> implementations.  It also documents other less common or outdated
> constructs.



>
>
>> 1.1.  History
>>
>>    The file URI scheme was first defined in [RFC1630], which, being an
>>    informational RFC, does not specify an Internet standard.  The
>>
>
> My personal preference is for documents to refrain from referring to
> their standards status or the status of other documents. "Status" is an
> ephemeral attribute that should be external to the details of a
> specification, IMO.
>
> That is, make the discussion be in terms of the technical and
> operational issues, not the standards status.
>
>
Ok. How about I just remove "History" altogether? It doesn't add anything
technical or operational.



>
>    definition was standardised in [RFC1738], and the scheme was
>>    registered with the Internet Assigned Numbers Authority (IANA);
>>
>
> IANA registration is a side-effect of the specs and typically isn't
> called out this way, I believe.
>
>
>    however that definition omitted certain language included by the
>>    former that clarified aspects such as:
>>
>
>    ... by former that...   missing word?
>
>
It says "...by the former that..."



>
>>    o  the use of slashes to denote boundaries between directory levels
>>       of a hierarchical file system; and
>>
>>    o  the requirement that client software convert the file URI into a
>>       file name in the local file name conventions.
>>
>
> *** Hmmm. A requirement like that moves this from being a URI
> specification to being a file protocol specification...
>
>
Thank you for saying that, you've triggered a bit of a light-bulb moment
for me about why I've had so much trouble getting this draft straight in my
head -- maybe it is actually a protocol spec. That said, I'd rather cut it
back to be a URI scheme spec. If we need to define the protocol in future,
then that's a future issue.



>
>>    The Internet draft [I-D.hoffman-file-uri] was written in an effort to
>>    keep the file URI scheme on standards track when [RFC1738] was made
>>    obsolete, but that draft expired in 2005.  It enumerated concerns
>>    arising from the various, often conflicting implementations of the
>>    scheme.  It serves as the spiritual predecessor of this document.
>>
>>    Additionally the WHATWG defines a living URL standard [WHATWG-URL],
>>    which includes algorithms for interpreting file URIs (as URLs).
>>
>
> How does it relate to the current draft?  At the least, doesn't it
> instead belong in the next 'Similar Technologies' section?
>
>
I was just listing other specs that roughly do what this one does. UNC
isn't the same as file:, but file: URLs are the same as file: URIs. That
said I could just wipe out the whole History section, then I'd probably
move the WHATWG reference to Similar Technologies.



>
>> 1.2.  Similar Technologies
>>
>>    The Universal Naming Convention (UNC) [MS-DTYP] defines a string
>>    format that can perform a similar role to the file URI scheme in
>>    describing the location of files.  A UNC filespace selector string
>>    has three parts: host, share, and path; see Appendix E.  This
>>    document describes but does not specify a means of translating
>>    between UNC filespace selector strings and file URIs in Appendix C.2.
>>
>
> "describes but does not specify" seems an odd distinction.  Not exactly
> sure what it means.
>
>
Normative vs informative; the appendices are non-normative descriptions of
things that are currently done, or that could be done -- but not things
that MUST be done (this comes back to Daniel's comment from 2014.)

I will work on the language, and work on clarifying the document structure,
since I don't want this confusion to be an issue.



>
>>
>>
>> Kerwin                    Expires June 3, 2016                  [Page 3]
>>
>> Internet-Draft                 file-scheme                 December 2015
>>
>>
>> 1.3.  Notational Conventions
>>
>>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>>    document are to be interpreted as described in [RFC2119].
>>
>>    Throughout this document the term "local" is used to describe files
>>    that can be accessed directly through the local file system.  It is
>>    important to note that a local file may not be physically located on
>>    the local machine, for example if a networked file system is
>>    transparently mounted into the local file system.
>>
>
> I'm not exactly sure what 'accessed directly' means, in the modern world
> of file systems.  Unfortunately, this is cast as a major point of
> concern in the document.  How is 'directly' different from 'no authority
> value is specified'?
>
>
Um, I mean something like "...can be access through the local file system
API without explicitly establishing network connections or engaging network
protocols." I don't know if that's any better.

I should also clarify that I'm talking about local files, not local URIs --
although they're equivalent. The definition of a local file URI is given in
section 3.

I'll try to rework it.



>
> 2.  Syntax
>>
>>    The file URI syntax is defined here in Augmented Backus-Naur Form
>>    (ABNF) [RFC5234], including the core ABNF syntax rule "ALPHA" defined
>>    by that specification, and importing the "userinfo", "host" and
>>    "path-absolute" rules from [RFC3986] (as updated by [RFC6874].)
>>
>
> Possibly cleaner:
>
>      [RFC5234]. Imported rules:
>
>         From [RFC5234]: ALPHA
>
>         From [RFC3986], [RFC6874]: userinfo, host, authority,
>                                   path-absolute
>
>
>    The core syntax in [RFC3986] includes "path" and "authority"
>>    components, for each of which only a subset is used in the definition
>>    of the file URI scheme.  The relevant subset of "path" is "path-
>>    absolute", and the subset of "authority" is "file-auth", given below.
>>
>>    Please note Appendix C that lists some other commonly seen but
>>    nonstandard variations.
>>
>>       file-URI       = file-scheme ":" file-hier-part
>>
>>       file-scheme    = "file"
>>
>>       file-hier-part = "//" auth-path
>>                      / local-path
>>
>>       auth-path      = [ file-auth ] path-absolute
>>
>>       local-path     = path-absolute
>>
>>       file-auth      = [ userinfo "@" ] host
>>
>
> Syntax problem:
>
>    If auth-path has no file-auth, since it's optional, then both the
> auth-path and local-path of file-hier-part reduce to path-absolute.
> Methinks that's creates a parsing ambiguity?
>
>
​I think we might have an operator precedence issue; I mean:

> file-hier-part = ( "//" auth-path ) / local-path​

​not:

> file-hier-part = "//" ( auth-path / local-path )

So auth-path and local-path can look the same, but are clearly
distinguished by the preceding double-slashes. I'll add parens to clarify
this (per guidance in https://tools.ietf.org/html/rfc2234#section-3.5 )



>
> Hmmm. I'm also not sure whether stylistic nuance and exact compatibility
> with the meta-specification documents this is based on are worth
> worrying about, but...
>
> I believe this could derive more smoothly from section 3 of RFC3986,
> with something like:
>
>    URI        = scheme ":" hier-part    ; from RFC 3986
>
>    scheme     = "file"
>
>    hier-part  = "//"
>                 ( authority
>                 / local-path )
>
> (The parens are to make the binding of the alternatives visually
> unambiguous. Relying on formal, implicit binding rules can create tricky
> usability errors. The hier-part rule in RFC 3986 might be an example:
> Concatenation is has tighter binding than alternatives, in ABNF, but the
> labeling in RFC3986 could imply a different parsing. Presumably the
> authors of RFC 3986 got the BNF correct, but I'm guessing a random
> reader could easily get it wrong... )
>
> ​
See above.



>    The syntax definition above is different from those given in
>>    [RFC1630] and [RFC1738] as it is derived from the generic syntax of
>>    [RFC3986], which post-dates all previous specifications.
>>
>
>  all previous -> the previous file URI
>
>
ACK



>
>    As a special case, the "file-auth" rule can match the string
>>    "localhost" or the empty string; either value is interpreted as "the
>>    machine from which the URI is being interpreted," exactly as if no
>>    authority was present.  To maximise compatibility with previous
>>
>
> 'as if no authority" was present.'  Probably should be 'were' present,
> but more significantly 'authority' is not defined.
>
> This is a reason to re-use the ABNF rulenames from RFC 3986, when you
> are importing them, but modify the <element> portion to suit the file
> scheme.  (Note that Ned Freed offers extended discussion about this,
> including going down a different line of resolution on this issue, in
> his 2 November 1:42pm posting. However the issue here is resolved, the
> main point is that the text needs to be consistent and clear in its
> language. It seems that the current text does not fully resolve the
> points he raised.)
>
>
I defined the authority using a reference to RFC3986, and then said that
the relevant syntactic subset is given as 'file-auth'. Is that not enough?
Also, "authority" is a word in its own right, not just a syntactic element
from RFC 3986. I use the word "path" throughout the text without defining
it, but nobody complains that there's no `path` in my ABNF.



> Since file-auth is shown as optional, that handles the empty string
> already and doesn't need a comment.
>
>
Hmm, yes, I suppose so. I was thinking that an element of the ABNF not
matched because it is [optional] is distinct from one not matched because
it's in an ( unused / alternative ) branch. I can fix it up easily enough,
I think, just by deleting a phrase.



>
> Offhand, I wonder whether the details of the "As special case" paragraph
> can (and should) instead be reflected directly in the ABNF?
>
>
I could change file-auth thus:

> file-auth = "localhost"
>​           / [ userinfo "@" ] host

(do I need parens again?)

Then the "special case" paragraph would just be a bit of text explaining
why "localhost" is its own special token. That said, I'm pretty sure I
already had that written at some point, and somebody complained that
"localhost" was redundant because it was already syntactically covered by
'host'. Should the ABNF lean towards a pure syntactic parser, or include
semantic chunks as well? (Asking for AD guidance here.)



>
>
> Kerwin                    Expires June 3, 2016                  [Page 4]
>>
>> Internet-Draft                 file-scheme                 December 2015
>>
>>
>>    specifications, implementations MAY choose to include an empty "file-
>>    auth".
>>
>>    Systems exhibit different levels of case-sensitivity.  Unless the
>>    file system is known to be case-insensitive, implementations MUST
>>    maintain the case of file and directory names when translating file
>>    URIs to and from the local system's representation of file paths, and
>>    any systems or devices that transport file URIs MUST NOT alter the
>>    case of file URIs they transport.
>>
>
> This is protocol language, not URI format/semantics language.
>
> I believe that the needs of this spec are served by something along the
> lines of:
>
> ​​
>     Some systems have case-sensitive file naming and some do not.  Hence
> the  file scheme supports case sensitivity, in order to retain the case
> as given. Any transport-related handling of the file URI scheme MUST
> retain the case as given. Any mapping to or from a case-insensitive form
> is soley the responsibility of the implementation processing the file
> URI on behalf of the referenced file system.
>
>
Sure thing, I'll work with that suggestion.



>
>
>
>> 3.  Operations on file URIs
>>
>
> Use of normative language in this section is inappropriate.  It does not
> provide protocol details and provides essentially no semantics.
>
>
​So, should the section even exist? Or should it just say: we're just
defining a scheme here, operations are protocol business.

BCP35 (http://tools.ietf.org/html/rfc7595#section-3.4) says that the scheme
definitions should define a default dereference operation. It doesn't seem
to account for schemes that aren't protocol-bound, though.



>    Implementations that provide dereferencing ooperations on file URIs
>>
>
>    ooperations -> operations
>
>
Yep, already noted and fixed.



>    SHOULD, at a minimum, provide a read-like operation to return the
>>    contents of a file located by a file URI.  Additional operations MAY
>>    be provided, such as writing to, creating, and deleting files.  See
>>    the POSIX file and directory operations [POSIX] for examples of
>>    standardized operations that can be performed on files.
>>
>
> What is the practical benefit of giving the reader superficial tutorial
> information about file system operations?
>
>
Well, I had to define what a file is earlier.

Does it not follow from the previous two sentences? I can remove it if it's
too bad, and point out the analogy between file operations and file URI
operations.



>
>    File URIs can also be translated to and from other, similar
>>    constructs, such as local file paths or UNC strings.
>>
>
> I don't know what this means.  And how is this sentence useful?
>
>
A file URI is a structured string, a file path is a structured string, and
a UNC string is a structured string. You can translate the former to and
from (one of) the latter.

In early version of the draft this translation was the only operation that
could be "performed on a file URI". We can remove it, but then should we
keep 3.1?



>
>    A file URI can be dependably dereferenced or translated to a local
>>    file path only if it is local.  A file URI is considered "local" if
>>    it has a blank or no authority, or the authority is the special
>>    string "localhost".
>>
>>    This specification neither defines nor forbids a mechanism for
>>    accessing non-local files.  See SMB [MS-SMB], NFS [RFC7530], NCP
>>    [NOVELL] for examples of protocols that can be used to access files
>>    over a network.  Also see Appendix C.2 for a discussion on
>>    translating non-local file URIs to and from UNC stings.
>>
>> 3.1.  Translating Local File Path to file URI
>>
>>    Below is an algorithmic description of the process used to convert a
>>    file path to a URI; see Section 4.
>>
>>    1.  Resolve the file path to its fully qualified absolute form.
>>
>
> What does this mean?  Where is it defined?
>
>
I guess it's short-hand for much more text about how some file systems have
concepts of relative vs. absolute paths and we have to use the absolute one
when such a thing exists. I'll think more about this one.



>>    2.  Initialise the URI with the "file:" scheme identifier.
>>
>>    3.  If including an empty authority field, append the "//" sigil to
>>        the URI.
>>
>
> sigil ???  pretty stylized vocabulary...
>
>
Well, it's a magical rune that doesn't mean anything outside of this
context. I've spent too much time in D&D (or possibly perl), sorry. Is
"token" better?



> What about the alternative of including a /non-empty/ authority field?
>
>
Hmm, yes, that's a point. If we want to keep this section, I should add
"localhost".



>    4.  Append a slash character "/" to the URI, to signify the path
>>        root.>
>>
>
>    5.  For each directory in the path after the root:
>>
>>        1.  Transform the directory name to a path segment ([RFC3986],
>>            Section 3.3) as per Section 2 of [RFC3986].
>>
>
> I think that Section 2 does not specify how to do a transform, although
> yes, it does make reference to doing one.  Rather, it specifies encoding
> rules.  The details of how to do a transform are left to the implementer.
>
> Hence I believe the above should be something like:
>
>      1. Transform the directory name to a path segment (RFC3986],
> Section 3.3]) to conform to the encoding rules of Section 2 of
> [RFC3986].  The specific rules for mapping between a file system name
> and a file scheme URI are outside the scope of this specification.
>
>
That works for me.



>        2.  Append the transformed segment and a delimiting slash
>>            character "/" to the URI.
>>
>>    6.  If the path includes a file name:
>>
>>        1.  Transform the file name to a path segment as above.
>>
>>        2.  Append the transformed segment to the URI.
>>
>
> A slash is required at the end of a directory, even if there is no file
> name?
>
>
If you're using it as a directory then yes. If you're using it as the
ultimate object (the "file") then no. We defined "file" as an "object" in
the file system earlier, which (going with the UNIX interpretation that
everything is a file) can include directories. As far as I know most
non-UNIXy systems around today can deal with this interpretation too.
Should I spell it out, or leave it up to interpretation?



>
> Differences from RFC 1738
>>
>
> Seems like this belongs elsewhere in the document, like maybe an
> appendix.  It is irrelevant except as an historical note.
>
>
I agree. I will look at moving it to an appendix.



>    In [RFC1738] a file URL always started with the token "file://",
>>    followed by an (optionally blank) authority and a "/".  That "/" was
>>    not considered part of the path.  This implies that the correct
>>    encoding for a file path in a UNIX-like environment would have been:
>>
>>         token     + authority + slash + path
>>       = "file://" + ""        + "/"   + "/path/to/file.txt"
>>       = "file:////path/to/file.txt"
>>
>>    However that construct was never observed in practice, and in fact
>>    would have collided with the eventual encoding of UNC strings in URIs
>>    described in Appendix C.3.
>>
>> 3.2.  Translating Non-local File Path to file URI
>>
>>    Translating a non-local file path, including a UNC string, to a file
>>    URI follows the same basic algorithm as for local files, above,
>>    except that the authority MUST refer to the network-accesible node
>>    that hosts the file.
>>
>
>    accesible  -> accessible
>
>
ACK



>    For example, in a clustered OpenVMS Files-11 system the authority
>>    would contain the node name.  Where the original node reference
>>    includes a username and password in an access control string, they
>>    MAY be transcribed into the userinfo field of the authority
>>    ([RFC3986], Section 3.2.1), security considerations (Section 6)
>>    notwithstanding.
>>
>>    See Appendix C.2 for an explicit handling of UNC strings.
>>
>>
>>
>>
>>
>>
>>
>> Kerwin                    Expires June 3, 2016                  [Page 6]
>>
>> Internet-Draft                 file-scheme                 December 2015
>>
>>
>> 3.3.  Incompatible File Paths
>>
>>    Some conventional file path formats are known to be incompatible with
>>    the file URI scheme.
>>
>> 3.3.1.  Win32 Namespaces
>>
>>    The Microsoft Windows API defines Win32 Namespaces [Win32-Namespaces]
>>    for interacting with files and devices using Windows API functions.
>>    These namespaced paths are prefixed by "\\?\" for Win32 File
>>    Namespaces and "\\.\" for Win32 Device Namespaces.  There is also a
>>    special case for UNC file paths in Win32 File Namespaces, referred to
>>    as "Long UNC", using the prefix "\\?\UNC\".
>>
>>    This specification does not define a mechanism for translating
>>    namespaced paths to or from file URIs.
>>
>
> No it doesn't, although it contains some language that almost seems to.
>  The language needs to be removed, in favor of the above simple sentence.
>
> Further, language about 'incompatibility' is a commentary that belongs
> in the Introduction or some other place outside the main spec.  It's an
> important discussion point, but it's not part of the spec.
>
>
Intro, or yet another appendix? I don't mind either way, it depends what
you think is best.



>
>> 4.  Encoding
>>
>>    To avoid ambiguity, a file URI SHOULD be transported as an
>>    Internationalized Resource Identifier (IRI) [RFC3987], or as a URI
>>    with non-ASCII characters encoded according to the UTF-8 character
>>    encoding [STD63] and percent-encoded as needed ([RFC3986],
>>    Section 2.5).
>>
>>    The encoding of a file URI depends on the file system that stores the
>>
>
> I'm not sure this sentence is correct.  It's a natural assumption but
> arguably one can lay any encoding convention on top of any data store.
>
>
I think this sentence came out of a discussion with Dave Thaler, although I
can't recall for sure (it appeared some time around September 2014.) What
you say is true, but it's only useful if that encoding is compatible with
(or can be translated to another encoding that is compatible with) the file
system's, so there's still a dependence. I mean, the whole point of this
paragraph is to say that you ought to encode it in UCS(+UTF8), even if the
file system doesn't use that.

Do you have a suggestion for something better to write, that isn't flirting
with untruth?



>    identified file.  If the file system uses a known non-Unicode
>>    character encoding, the path SHOULD be converted to a sequence of
>>    characters from the Universal Character Set [ISO10646] normalized
>>    according to Normalization Form C (NFC) [UTR15], before being
>>    translated to a file URI, and conversely a file URI SHOULD be
>>    converted back to the file system's native encoding when
>>    dereferencing or translating to a file path.
>>
>>       Note that many modern file systems encode directory and file names
>>       as arbitrary sequences of octets.  In those cases, the
>>       representation as an encoded string often depends on the user's
>>       localization settings, or defaults to UTF-8 [STD63].
>>
>>    When the file system's encoding is not known the file URI SHOULD be
>>    transported as an Internationalized Resource Identifier (IRI)
>>    [RFC3987] to avoid ambiguity.  See Appendix D for examples.
>>
>
> ​​
> I'm inclined to think that this section either needs to be far more
> complete -- and I'm not recommending it do that -- or it merely needs to
> caution implementers to make sure that file scheme URI storage needs to
> be idempotent with the original, interoperable form.
>
>
A lot of this is just quoting or paraphrasing RFC 3986. I could probably
replace​ a bunch of it with a reference, but that would take some editorial
effort on my part, so I won't attempt it immediately. This goes back to
Dave Thaler's comments about encoding file URIs and how he wants them to go
away completely because of it. (The IRI bit definitely comes from him,
filtered through my interpretation.)



>
>
>> 5.  Origins
>>
>>    As per [RFC6454], Section 4, when determining the origin of a file
>>    URI implementations MAY return an implementation-defined value.
>>
>
> Now we are back to protocol, rather than basic representation. But it
> seems minimal in detail and utility.
>

mnot asked how 'file:' fits into the web's origin model; this sentence is
pretty much the answer. If it doesn't belong here, then I'm not sure how to
answer the question.



>
>>
>>
>> Kerwin                    Expires June 3, 2016                  [Page 7]
>>
>> Internet-Draft                 file-scheme                 December 2015
>>
>>
>>    Historically, user agents have granted content from the file URI
>>    scheme a tremendous amount of privilege.  However, granting all local
>>    files such wide privileges can lead to privilege escalation attacks.
>>    Some user agents have had success granting local files directory-
>>    based privileges, but this approach has not been widely adopted.
>>    Other user agents use globally unique identifiers for each file URI,
>>    which is the most secure option.
>>
>
> This paragraph belongs in the next, Security Considerations section.
> Without the protocol-ish paragraph before it.
>

This same thought occurred to me yesterday. I'll move it. It means the
final sentence need to be fixed, though, since that's talking about
"origin" identifiers.



>
>
>> 6.  Security Considerations
>>
>>    There are many security considerations for URI schemes discussed in
>>    [RFC3986].
>>
>>    File access and the granting of privileges for specific operations
>>    are complex topics, and the use of file URIs can complicate the
>>    security model in effect for file privileges.  Software using file
>>    URIs MUST NOT grant greater access than would be available for other
>>    file access methods.
>>
>
> This sort of normative statement has no real meaning, and certainly none
> without explanation.  At base, the reader cannot tell was satisfies the
> normative statement and what does not.
>
>
Yes, I agree. This is really old language (possibly predating my draft, or
maybe suggested to me early on, I can't quite remember). Is it good enough
to get rid of this sentence, and move the paragraph from above to here
(since they address the same thing, modulo the term "user agents")?



>
>    File systems typically assign an operational meaning to special
>>    characters, such as the "/", "\", ":", "[", and "]" characters, and
>>    to special device names like ".", "..", "...", "aux", "lpt", etc.  In
>>    some cases, merely testing for the existence of such a name will
>>    cause the operating system to pause or invoke unrelated system calls,
>>    leading to significant security concerns regarding denial of service
>>    and unintended data transfer.  It would be impossible for this
>>    specification to list all such significant characters and device
>>    names.  Implementers MUST research the reserved names and characters
>>    for the types of storage device that may be attached to their
>>    application and restrict the use of data obtained from URI components
>>    accordingly.
>>
>>    Additionally, as discussed in the HP OpenVMS Systems Documentation
>>    <http://h71000.www7.hp.com/doc/84final/ba554_90015/ch03s09.html>
>>    "access control strings include sufficient information to allow
>>    someone to break in to the remote account, [therefore] they create
>>    serious security exposure."  In a similar vein, the presence of a
>>    password in a "user:password" userinfo field is deprecated by
>>    [RFC3986].  As such, the userinfo field of a file URI, if present,
>>    MUST NOT contain a password.
>>
>
> Is there really no stable, published document that says something similar?
>
>
​Not that I could discover.​



> ​<snip>
>>
>
I've just spent most of a day not working on my day job and my family is
wondering where I am, so I'll leave the comments at that for now. Some of
the incorporated/draft changes to my working copy are online now:
http://phluid61.github.io/internet-drafts/file-scheme/

Cheers, and thanks again.
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/