Re: [nfsv4] review of "draft-ietf-nfsv4-layrec-00"

[Jeff Layton <[e-mail]>]

On Fri, 2024-03-01 at 06:34 -0800, Rick Macklem wrote:
> On Fri, Mar 1, 2024 at 6:17 AM David Noveck <[e-mail]> wrote:
> > 
> > 
> > 
> > On Thursday, February 29, 2024, Jeff Layton <[e-mail]> wrote:
> > > 
> > > Most of this document looks fine to me, but it has this statement in
> > > section 2:
> > > 
> > > "The metadata server MUST NOT have been resilvering the file such
> > > that it has a different layout (set of mirror instances) than the
> > > client before the restart of the metadata server."
> > 
> > 
> > I also had problems with this statement but my reaction is different.  See below.
> > 
> > > 
> > > 
> > > The rationale for this seems clear,
> > 
> > 
> > The rationale is clear, but not how to enforce this constraint.  Given the use of the perfect aspect here, it seems you'd need a time machine to go back to prevent a resilvering that started before the MDS restart or a way to absolutely prevent MDS restart, which seems impossible.
> 
> I'll note that for the FreeBSD pNFS server, the DS is taken offline
> when any resilvering
> is done. ie. Layouts are all recalled, then it is disabled (ie. no
> layout for the DS are
> issued) and the DS is not brought online until the resilvering is done.
> This resilvering and bringing the DS online is a manual operation done
> by a sysadmin.
> So, for the FreeBSD pNFS server, it is DS that is absolutely prevented
> from restart,
> to use David's terminology.
> 
> If the DS remains online, I think it comes down to "the layout cannot be changed
> at all due to resilvering" which is a somewhat stronger statement than
> the above.
> What do others think?
> 
> > 

(readding Tom to cc list)

I think it has to be more than just the layout not changing. Even if the
layout doesn't change, I don't see how you can safely resilver the
mirror while the client could potentially be writing to it. It would be
rather easy to get a "torn mirror" that way.

OTOH, I am leery of putting in too much in the way of MUST NOT
directives, as we occasionally find ways to safely do things we might
originally think must be forbidden.

It is a bit of a tightrope.  ;)


> > Perhaps something else is intended here?
> > 
> > > 
> > > 
> > >  but it seems like resilvering the
> > > file while the clients hold any sort of RW layout would be a bad idea,
> > > no?
> > 
> > 
> > Yes, but rfc2119 does not define "A BAD IDEA".
> > 
> > It appears resilvering is not valid unless it can be known that there are no outstanding RW layouts.
> > 
> > > 
> > > 
> > > In any case, adding another sentence or two explaining this requirement
> > > (even if it seems obvious) wouldn't go amiss.
> > 
> > 
> > I'm not sure how much is needed but it needs to be clearer if this is really a requirement, and, if it is, how that requirement is to be enforced
> > 
> > > 
> > > 
> > > Aside from that nit, the draft seems fine to me!
> > > 
> > > Cheers,
> > > --
> > > Jeff Layton <[e-mail]>
> > > 
> > > _______________________________________________
> > > nfsv4 mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/nfsv4
> > 
> > _______________________________________________
> > nfsv4 mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/nfsv4

-- 
Jeff Layton <[e-mail]>