Re: [nfsv4] More layout stuff: layout return on close

[<[e-mail]>]

Let me just say I have to say with no quoting.  Once you have five sets
of greater-than's, it just gets impossible.

We're talking about scrapping RFC5661?!  What I mean is Trond is talking
about that, and we do feel his pain :-)  But still, it seems a bit,
well, extreme.

My interpretation was (and maybe there is something in the spec which
contradicts this) or it is in some way wrong is:

     The purpose of the seqid in the CB_LAYOUTRECALL
     stateid is to define what version of the layout
     state corpus being referred to, since, if 
     you the client don't know, you can't know what
     I (the server) am talking about.

     So if you have a LAYOUTGET or return in flight
     and the CB_LAYOUTRECALL is beyond the one I have
     received (i.e. it is more than one more than the 
     last one I have received), I cannot process the 
     recall and should wait until I can.  

     If I don't do that, my no-matching-layout is 
     of no value because it may mean that I just haven't
     seen the layout that was recalled yet and so the 
     server could not deduce anything from my 
     no-matching-layout response.  Me being forgetful
     is different from me lacking precognition :-)

     As has been noted, the CB_LAYOUTRECALL doesn't 
     change state, so it doesn't really make sense for
     it to change the seqid.  I know this seems wrong to
     Fred, and maybe it is wrong, but if it is, I think 
     we need a stronger argument, of the form "If you 
     don't do this, then this case breaks".

     The seqid sequencing applies to state transitions
     that are made on the server (and provides for ordering
     those successive versions of *server* layout state).
     So, having somebody other than the server change the
     seqid, seems wrong to me.  Maybe I'm wrong.

     If the server and the client are both changing the 
     seqid, I need good strong drink, before somebody
     explains to me how that can result in anything but
     chaos.  (Just one drink;  I really will listen).

It seems to me from the stuff I have heard recently is that many of the
problems are not in the protocol per se (i.e. it is not full of
mistakes), but rather in the explanation, which does need work to be
clearer.  I certainly could be proven wrong.

 

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf
Of Trond Myklebust
Sent: Tuesday, November 30, 2010 5:11 PM
To: Fred Isaman
Cc: Benny Halevy; [e-mail]
Subject: Re: [nfsv4] More layout stuff: layout return on close

On Tue, 2010-11-30 at 10:39 -0500, Fred Isaman wrote:
> On Tue, Nov 30, 2010 at 10:26 AM, Trond Myklebust
> <[e-mail]> wrote:
> > On Tue, 2010-11-30 at 10:20 -0500, Trond Myklebust wrote:
> >> On Tue, 2010-11-30 at 17:14 +0200, Benny Halevy wrote:
> >> > On 2010-11-29 01:23, [e-mail] wrote:
> >> > > I think these two case are different.
> >> > >
> >> > > I think the CB_LAYOUTRECALL case does have sequencing issues
but they
> >> > > are dealt with, at least in the case of LAYOUTRECALL4_FILE, by
the fact
> >> > > that that the seqid in lor_stateid tells the client what
revision of
> >> > > layout state is being referred to, and thus implicitly tells
the server
> >> > > what layout changes in flight are or are not covered by the
> >> > > NFS4ERR_NOMATCHING_LAYOUT.
> >> > >
> >> > > As Benny points out, the CLOSE doesn't have that, but I don't
see an
> >> > > ordering problem.  The seqid ordering mechanism is to make sure
that the
> >> > > client and the server agree on the corpus of layout-segments
with
> >> > > return-on-close best thought of as attribute for each segment
part of
> >> > > that layout segment corpus.  The close doesn't change that
corpus in a
> >> > > way that interferes with ongoing LAYOUT operations.  It merely
> >> > > eliminates from active use those bearing the return-on-close
attribute
> >> > > and that shift can be done independently on the client and the
server.
> >> > >
> >> > > Let's get more concrete here.  I don't see how there should be
ordering
> >> > > issues between close and LAYOUTRETURNs.  If the segment is
returned
> >> > > before the close or after the close, it is still gone.  I don't
see any
> >> > > perceivable difference.  Now let's take the case of LAYOUTGET
and close.
> >> > > Suppose you get a LAYOUTGET response, and it overlaps (for
simplicity
> >> > > let use say it is identical but I don't think the overlap case
is really
> >> > > any different).  What determines whether the client should
treat this as
> >> > > a segment he still has (post-close), is whether it has the
> >> > > return-on-close flag set.  If it does, he should assume it is
gone (and
> >> > > it must have been sent before the close was processed).  If it
doesn't
> >> > > have that set he should assume it still exists, apart from
ordering.  If
> >> > > it was sent before the close, then the layout segment was
upgraded to
> >> > > non-open-specific, so it doesn't go away at close.  OTOH, if it
was sent
> >> > > after the close, then the segment went away at the close and
was added
> >> > > afterwards.  However, in either case, he (the client) should
assume he
> >> > > has still has it and the that server matches in that regard.
> >> >
> >> > How about the resulting layout stateid?
> >> > What should the client use on the next layout operation?
> >>
> >> The same layout stateid he would have used if the server had
recalled
> >> that particular layout segment and he'd replied NOMATCHING_LAYOUT.
> >>
> 
> The CB_LAYOUTRECALL supplies a stateid.  Are you suggesting that the
> client should not update its seqid to that of the recall when it
> replies NOMATCHING_LAYOUT?  Because that seems wrong.

Oh crap! You're kidding me....

We invented a whole new damned convention for bloody stateid seqids ONLY
for use in the damned CB_LAYOUTRECALL?

I've got a better idea. Let's scrap RFC5661 and start over again. This
time, we pay attention to the existing conventions for stateids. One of
those conventions is that if the seqid changes occur _only_ when a state
change occurs.

No wonder we're having problems...

> >> IOW: the most recent stateid returned by LAYOUTGET, unless the
client
> >> has forgotten all layout segments for that file (i.e. the whole
layout),
> >> in which case it needs to use the open stateid.
> >
> > So, the thing to remember here is that the layout stateid is only
useful
> > for ordering LAYOUTGET w.r.t. LAYOUTRETURN. It has no other purpose.
> >
> 
> It is also for ordering of LAYOUTGET wrt CB_LAYOUTRECALL.  For all
> intents and purposes, a return of NOMATCHING_LAYOUT to a
> CB_LAYOUTRECALL is a LAYOUTRETURN, and ordering needs to be paid
> attention to.

Right, but there is no state transition for a layoutrecall. Just like a
delegation recall, it leaves the state unchanged as far as the client is
concerned...

IOW: if we had made the damned LAYOUTRECALL take a stateid with the
_same_ seqid as the most recent LAYOUTGET that is being recalled, then
we wouldn't be in this ridiculous debate.


The point is: as far as the client is concerned, forgetting a layout
segment is not a seqid-changing state transition. Apparently we have to
do the seqid change on LAYOUTRECALL, but that is an aberration due to
apparent blindness on the part of the spec reviewers (of which I was
one, yes). There is no reason to introduce the same insanity everywhere
else.

Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[e-mail]
www.netapp.com

_______________________________________________
nfsv4 mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/nfsv4