Re: [apps-discuss] "local convention" in draft-kerwin-file-scheme

[Matthew Kerwin <[e-mail]>]

On 30 September 2014 04:35, Sean Leonard <[e-mail]> wrote:

> Colleagues:
>
> We ought to step back for a moment and ask ourselves what exactly the
> goals are of getting a common file URI scheme through the IETF process.
>
>
I agree. I can say, from the start, my original reason for taking on the
draft was to address a single use-case: there are a bunch of really cool
"URI access" libraries out there, and people like to use those to access
local files[1]. So, the goal is: a means of specifying file names that is
compatible with the generic syntax of RFC 3986, and can be common across
all the "URI access" libraries and other similar consumers.

[1]: https://bugs.ruby-lang.org/issues/8544



> ​
> Ultimately we are all here to promote "interoperability". There is a lot
> of software out there that needs to identify local resources in the OS file
> system in the same data type (URI) as Internet resources. The way to
> distinguish these local resources in the URI way is--at a minimum--to tack
> on "file:".
> ​
> ​
>
> ​​
> File system resources are normally addressed by the operating system's
> convention. We can come up with all sorts of notations, but at the end of
> the day the OS system call (POSIX open, Win32 CreateFile, etc.) is what
> matters. The OS needs to be consistent with itself and with applications on
> it; it's also important that two instances of the same OS (or OS families)
> agree on things, so that applications and data written with that OS
> (family) in mind can work on different machines.
>
> But from the perspective of Internet architecture, all those things are a
> "local convention"--that magical escape that in IETF-speak means "do
> whatever at the endpoints, it doesn't affect the network".
> draft-kerwin-file-scheme talks about local conventions but the overall gist
> is trying to stuff local conventions into a common format, and flailing
> about by addressing these disparate use cases (mostly Windows, but some
> OpenVMS) that have cropped up.
>
>
The apparent "flailing" could just be me not being able to express myself
very well. The path-to-URI conversion is really simple, the only weirdness
being the various conventions for file system roots. Converting back can be
trickier to specify, so it's more tempting to hand-wave there.

But aside from that detail, the interoperability I was aiming for only goes
as far as the syntax. The only operations/methods I explicitly address are
parsing, and converting to and from file paths (once you've done that it's
not a file URI anymore, so of course the local system's syscalls are
appropriate), and the conversions are necessarily system dependent.



> I would like to suggest a totally different approach:
> 1. file: identifies local filesystem resources (this includes filesystem
> resources accessible via network paths such as UNC, since the filesystem
> facility of the OS accommodates them).
> 2. # is the fragment.
> 3. everything else (the RFC 3986 <hier-part> and <query> productions) is a
> local convention.
>
>
1. Yep.

​2. I originally removed the fragment because fragment semantics depend on
the content-type of the addressed resource, and files don't have a
content-type. I kept it off because as far as I'm aware no file system
conventions include fragments (or equivalent semantics), and the as only
operations are converting to/from file system conventions, any fragments
would be immediately lost anyway.

3. Pretty sure this breaks the driving goal, which is: can be parsed by any
generic URI parser. Unless you mean to introduce local URI conventions
that, while compatible with RFC 3986, might not be compatible with other
local URI conventions. That ... offends me in some way I can't well
articulate. <https://www.youtube.com/watch?v=5tkM6Zfr47o>



> The draft can still include all of the current content, but move it into
> informative appendices.
>
> If an application encounters a file URI for a local filesystem resource,
> it should deal with it using the local convention. If an application
> encounters a file URI for some non-local filesystem resource, it should
> *leave it alone*. The end.
>
>
Again, this is all what happens after the conversion.​ It doubtless comes
across as a bit weak when I keep saying, "I don't care what you do with it
once you've converted it to a file path", but this whole discussion
reinforces to me the fact that we can't reasonably define any more methods.
They all fall short.



> In other words: why bother parsing a file URI if you know it's not going
> to refer to a resource that you can make use of? If you're on Ubuntu 14.04,
> why do you care about interpreting a file: URI that's meant for Windows 7?
> The only thing this does is invite security problems, because attackers
> will tell you (via the network) to perform operations on local resources
> that they don't have the authority to say anything about.
>
>
Catch 22: how can you know it's not going to refer to a particular resource
without parsing it? And I could try to tell you, on your Ubuntu 14.01
system, to dump /etc/passwd without resorting to Windows drive letters.



> Another issue is the matter of non-ASCII character encoding, which has
> always been dicey. [POSIX] is very clear that a paths as a sequence of
> *bytes*, not characters. Other than "/", <NUL>, and the relative pathnames
> "." and "..", it's anything-goes. The mapping between bytes and characters
> depends on the locale. In Windows NT, the kernel treats pathnames as
> Unicode (specifically UCS-2/UTF-16); translations are done in user-space.
> In Mac OS X, the HFS Plus file system converts all file names to decomposed
> Unicode, i.e., Normalization Form D [HFSPLUSREF].
> ​
> ​
>
>
Yep, I'm on board here (arguments about how accurately OS X maps to NFD
aside), and this is the bit I want most to get right.



> It is kind of seen as a moral imperative in the IETF to use UTF-8 (and
> Normalization Form C), possibly because of [BCP18] and [RFC5198]. But here,
> the only thing common is diversity--and each operating system (vendor) has
> its own good reasons for its individual decisions. Since file: URIs are
> always stored *in context*, context (the operating system documentation,
> the LC_ALL environment variable, the declared encoding of the text
> document, the identity of the local machine, etc.) provides all the info
> that you need. Just say that the file: URI (outside of the fragment
> component) encodes with local conventions--which are provided by
> context--and be done with it.
>
>
I think the moral imperative is to provide interoperability, and when all
else fails, Unicode with NFC is a good (or at least convenient)
fallback/default.

Regarding encoding, I took​ my lead from Microsoft and went as far as
defining conversions to IRIs (which are NFC Unicode, and thus unambiguous
and universal, but not necessarily UTF-8), and said that you /can/ go the
next step and convert them to URIs (if you want.) In draft 13 there's the
beginnings of a reason to leave it at IRIs, sketched out as examples.



> If you are taking things like file: URIs out of context...three
> suggestions:
>
> 1. Translate the file URI into the destination context. For example: if
> you are in an HTML editor and you really want to reference
> <file:///D:/pics/catpic.jpg>, but you are saving the HTML document in a
> workgroup share, change the reference to <file://mycomputername/pics/catpic.jpg>,
> and ensure that D:\pics is shared as \\mycomputername\pics with appropriate
> access. Better yet: upload the resource to an Internet-accessible location
> such as <http://i.imgur.com/ziZNc2s.jpg>, which for purposes of Internet
> architecture, provides a more universal context.
>
> 2. Dereference the file URI and put whatever content you need into the
> context. For example: in an HTML e-mail, don't use <img
> src="file:///D:/catpic.jpg">; use <img src="cid:catpic93298@myemail"> and
> put the cat pic in the e-mail per [RFC2392] and [RFC2387].
>
> 3. Don't do that. It doesn't make sense.
>
>
​That's a different use-case: writing one URI that can be consumed
anywhere. All I want is one syntax that can be used to write or parse a URI
anywhere.

I agree that if you *do* want to share a file you either make it accessible
to everyone who will receive the URI (e.g. SMB), or use a different scheme
(either n:1 like ftp/http/etc, or n:n like cid/data/...).

At our university, central IT has set up lots of SMB shares and mapped
drives, etc., so it's very convenient to email links to, e.g.
<file:///i:/section/minutes/some-meeting.doc> because we all share the same
I-drive, and because Outlook lets us interact with those links "properly."
I just want to make sure that the one guy who insists on using Thunderbird
doesn't miss out because Thunderbird has a different parser.

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/