RE: [nfsv4] Some stuff on referrals/migration

Brent Callaghan wrote: 
> Given that fs_locations has been in the spec for several years
> now, and we're still trying to figure out how to implement it,
> is it time to re-evaluate it ?

I think I'll take issue with the phrase "still trying".  You 
make it sound like we have been slaving away trying to implement 
this feature and been unable to do it.  If that were the case, 
then there might be a case for drastic action.

I think the reality is rather different.  In my case I spent
several years ignoring the whole area as I implemented other 
features.  I then spent less than a week hacking up a 
migration prototype (no referrals) which was tested a little
bit at a bakeathon with Rob Thurlow's prototype client support.
When I realized that referrals are really a sub-case of 
migration, I spent half a day in the airport and on a
plane reading the spec carefully and thinking about the spec 
details and writing up the core of the notes I sent out.  I 
then spent several months putting off completing those, and 
finally spent an hour or two finishing those notes and then 
sent them out.  Given that Netapp's review season is coming 
up, it might make sense to see if I can work up an appropriate 
level of conviction when delivering the line "And I spent a 
ton of time working on implementing that damn migration feature", 
but given that this is a public forum I guess I've just blown 
that opportunity :-).

I certainly would have liked it if we had been able to get this
implemented faster, but people have been busy implementing other
features first.  If more implementation resources were available,
then we could still progress on migration/replication while doing
everything else.  But I don't make decisions on overall v4 
resource levels and I suspect that nobody else on this list does
either.  So implementation of migration and replication seems 
destined to trail behind. 

> The NFS4ERR_MOVED error looks useful, but the work
> required to extract fs_locations data seems very complex.

At times I considered the possibility of having this information
come back with the MOVED error, making the error returns a
big switched union where errors like MOVED could come tied
together with any necessary additional information.  I've
come to the conclusion, however, that RFC 3530 basically
took the right path (so to speak :-), and that the problems
are in the way the details are specified.

You have to separate issues of implementation complexity from 
those of spec complexity.  As far as implementation complexity,
going out and doing a GETATTR is more complex than having the
needed information just handed to you but if you really have
problems issuing a GETATTR and extracting the results, you
have bigger problems than dealing with migration.

As far as spec complexity, there is an issue in that we have
the odd situation of you accessing some attributes on a
file system which isn't there, which does seem kind of weird.
The spec deals with this inelegantly by making fs_locations a
big special case.  Looking at this in detail I've decided that
RFC3530 made it a bit too special and that the concept that
we can return some (small set of) attributes for an absent
file systems is best generalized since things like 
mounted_on_fileid and fsid make sense to return even for 
an absent fs.  I think generalizing the concept is the 
best way to "domesticate" the weirdness.

> NFS users already use referrals via global paths in symbolic
> links, e.g. "foo -> /net/otherserver/foo". They can do this
> today with NFSv2 and NFSv3. Symlinks have the nice property
> that they can be manipulated as filesystem objects.  It's
> not clear how an fs_location attribute would be represented
> on a server.

How something is represented on the server is not something
we want to specify.  How you represent things in your fs and
we represent them in ours will be different, even though we
store the same information.

The issue is how things are represented in the protocol.  Both
symlinks and fs_locations have representations in the protocol.
The big issue with using symlinks is that they are not free
for use between the client and server.  In systems which 
support them, they are visible to applications so that a
symlink is not the equivalent of crossing a mountpoint.  

It is true that we have no way to have (privileged) users
instruct the filesystem on such matters as replication
and referral configuration.  That might be a worthwhile
thing to pursue if there is interest.  You could create
a new kind of symlink-like object, a "junction" say, which
clients would not export to applications.  However, you 
still have the issue of migration to consider.  If there
is the possibility of migrating a file system while in use
(and RFC3530 does provide for that), then you have to tell
the client where it went.  The way that RFC3530 specifies
to do that is via MOVED and fs_locations.  It is unclear
how symlinks or any variant can provide that functionality,
although they might be able to provide something that could
address the sub-case of referral.  The crucial point for the
protocol is precisely that referral is a sub-case of 
migration.  That is, if you provide a way that clients A and
B while accessing file X can be told that it is now somewhere
else, you wind up also having to be able to tell client C
which is just a bit slower that same thing.  Do you want to
do that latter using the same basic mechanism or a completely
different one?  Unless you are prepared to remove migration
and replication support from the protocol (and I vote "No"
to that), then referral follows naturally, although, as I
have said previously, the spec needs some work to clarify
the issues. 

> It seems that you could implement most of fs_locations in
> a symbolic link, perhaps by embedding an NFS URL in the
> link text.

Most???  I can't see that now matter how ones look at it.

Fs_locations performs three functions: replication, migration,
and referrals.  The symbolic link could address one out of
three (kind of badly, I think).

Another way of looking at things is that replication and
migration are the two basic pieces of functionality and referrals
are a sub-case (and indeed they really are a sub-case in that
if you support migration you have to support referral to deal
with the case of a client that happens to first reference the
fs just after it moves).  So a symlink-based approach could
deal with only one sub-case of the two basic functions of 
fs_locations.

> Would it be better to explore this further?

Not as far as I'm concerned but this is the sort of thing
on which opinions will differ.

As I understand it, what you are proposing (or thinking about
proposing) is making fs_locations (and NFS4ERR_MOVED?),
mandatory-to-not-implement in v4.1, perhaps with some 
replacement functionality.  If you think that's the right 
way to go, then you should think about formulating a detailed 
proposal that we can discuss in Ann Arbor. 

_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4