RE: [nfsv4] Some stuff on referrals/migration

["Noveck, Dave" <[e-mail]>]

As I remember things, we had a discussion of global namespace
as a possible work item for v4.1.  The conclusion was that 
the referrals that were in v4.0 were just a small part of
what was needed for this.  It didn't look like adding what
was needed was a small job and nobody stepped up and said
"I'll make it happen", so the subject was dropped.

I do think AFS with a common implementation had a big
advantage here, but I don't think this is out of the question
for NFS-v4.  It ain't going to happen for v4.1, but with
such a big item (and it is big), I think it is best to
take scheduling out of the equation.  I think it is 
reasonable to work toward a coherent description of how 
this will be done in NFS-v4, with the expectation that it 
will go into whatever v4.x is currently open for business, 
when it is ready.  Let's not propose it for each successive
v4.n and decide there is no time, for that cycle, and then
repeat the exercise for v4.n+1, etc.

Let me go into sales mode for a second.  This is a valuable
feature that people just love to have.  It makes sense for
it to make the transition to a mainstream open protocol.
I realize that people with the knowledge (of AFS particularly) 
and skills to do this work have many demands on their time, 
but I urge them to consider the value that this would bring 
to NFS-v4 and its users.  I think discussing this at Ann 
Arbor is a waste of time, if there are no volunteers to do 
the work, but if you are prepared to sign up for this, it 
makes sense to tell Spencer and we can schedule a discussion 
of global namespace as an agenda item for the 6/9 meeting.

As to,

> There are some issues that NFSv4 cannot have its own way,
> and filesystem relocation is probably one of them.

It can't dictate to other protocols how (or even if) they
handle the issue.  

> How
> do you make it work on a multi-protocol NAS server ?

There are two models.  Let me first talk about the every-
protocol-for-itself approach.  Actually, this issue is
on the simpler end of the spectrum, as multi-protocol
issues go.  In other cases, you have to deal with the
interactions among multiple protocols.  For example, when
a CIFS or v4 client makes a share reservation denying WRITE,
you have to handle writing by v2/v3 clients who have no
concept of denial because of a share reservation.  So you
are forced to map this to a permission error, which isn't
perfect but nothing is perfect ..

Having each protocol deal independently with an external 
event is inherently simpler which doesn't mean that it is 
simple.

> How do the CIFS, AFP and NFSv2,v3 clients follow a filesystem
> that's been moved ?  

Microsoft has something called DFS (not to be confused with
what other people call DFS) which I think does something
referral-like.  As far as migration of an actively used fs, 
you can just forget it.  In CIFS, session and connection 
state are tightly bound together and I don't see how you 
can disconnect here and reconnect there sanely.  It is the 
same with a server reboot.  You disconnect and reconnect, 
the applications see the event and handle it as well as they
can.  If you don't like it, don't use CIFS.

NFS allows reclaim after server reboot.  The fact that CIFS
doesn't does not mean that that feature is valueless, even
though it does nothing for CIFS users.  The same thing applies
with v2/v3 vs. v4.  With v2/v3 you can use symlinks and 
automount to do something referral-like, but if the fs moves
to a different server, you are going to get STALE and remount.
That isn't nice, but it does not indicate, to me at least, that
v4 should not do better.

> I just don't think file server admins
> are going to move stuff around - unless there's some kind
> of accomodation for existing NFS clients.

In the case of migration while a filesystem is being used, 
the only accommodation that I can see would be proxying, 
which starts to shade over into the true-multiprotocol
approach, discussed below.

The way I see it, is that admins are moving things around now.
They just don't like the consequences.  Implementation of v4
migration is simply a way of reducing their pain, and that's a
good thing.  I think it is true that admins will not move
things around much more freely until farther in the roll-out
of v4.  Until the majority of their (important) clients are
v4 with migration support, the benefit of the feature, where
it is available, will be in pain reduction, rather than
enhanced freedom to move things around.

The only approach I know of to handle migration in a true
multi-protocol fashion (as opposed to every protocol for 
itself within a multiprotocol framework), is the Spinnaker-
type approach.  Clients talk to servers which are in essence
proxies for the servers which actually contain the data.  The
entire cluster of proxies plus data servers appears to the
clients as a single system so that migration of data does 
not happen as far as the clients are concerned, and this
includes NFS and CIFS clients (although no v4 yet).

A number of protocols have been developed on a proprietary
basis to do this sort of thing.  The proxies speak this protocol
to the back-end data servers and you get scalable access to
a single namespace as well as the freedom to migrate data
betweeen data servers.

It is at least conceivable that you could develop such a 
protocol on an open basis but it would be a *big job* (in 
72-point extra bold type).  You would need a protocol for 
the communication between the proxy and the data server 
which is capable of faithfully representing the semantics 
of NFS v2/v3/v4, CIFS (in all its perversity.  Yeah, yeah, 
I'm biased, but if you think I don't like CIFS, talk to the 
people that work on it), and whatever other protocols you 
want to support. My estimate is that it is about 5-10 times 
the size of that v4, in terms of effort, so I'm not clear how
it could be done since I would think that an 40+ year effort 
is out of the question.   
  

-----Original Message-----
From: Brent Callaghan [mailto:[e-mail]]
Sent: Thursday, May 20, 2004 10:02 PM
To: Noveck, Dave
Cc: [e-mail]
Subject: Re: [nfsv4] Some stuff on referrals/migration


Hi Dave,

I think we had a referrals/migration discussion a year or so
back, where there issue came down to some fundamentals of a
global namespace: should it be implemented at the client or
at the server.  Currently it's based at the client, via automounters,
and NFSv4 referrals are an attempt to base it at the server.
But I don't think we're going to resolve that here.

I think NFS4ERR_MOVED and fs_locations was proposed with a more
modest goal of allowing server admins to shuffle NFS filesystems
around and the back end and not have to tell the clients, i.e.
the clients would just automagically follow the referrals,
mounts could stay in place, and files stay open.  I believe
it was inspired by the AFS filesystem, which could quite
easily relocate filesystems on the fly via some clever
snapshotting and simple Volume Location Database updates.
However, I don't think NFS is anywhere near as monolithic
as AFS was, with its VLDB, cells, and special filesystems.

There are some issues that NFSv4 cannot have its own way,
and filesystem relocation is probably one of them.  How
do you make it work on a multi-protocol NAS server ?
How do the CIFS, AFP and NFSv2,v3 clients follow a filesystem
that's been moved ?  I just don't think file server admins
are going to move stuff around - unless there's some kind
of accomodation for existing NFS clients.

	Brent


_______________________________________________
nfsv4 mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/nfsv4