Re: [nfsv4] Review of draft-haynes-nfsv4-delstid-00

[Trond Myklebust <[e-mail]>]

Hi Tigran,
I think both these proposals are interesting, but I agree with Tom that
there is a problem with any proposal that requires rewriting
applications in order to make use of the APIs being exposed. Unless you
are just going to use FATTR4_ACCESS_LATENCY in the same way that you
would use an 'online/offline' state variable, you would need to get
user or application level input in order to determine a latency
acceptance criterion. You would then need application policy about what
to do if the server exceeds that latency acceptance (e.g. due to tape
robot congestion or cloud archiving service level agreements)...
Ditto for the OPEN4_ONLINE_ONLY flag; that is not supported by any
POSIX API, so we'd also have to rewrite applications specifically for
that interface.
That said, both suggestions could be used to implement our use case, so
if you have a compelling use case that requires these enhanced APIs
then we can work with that.
Cheers   Trond
On Tue, 2018-07-24 at 08:17 -0700, Tom Haynes wrote:
> Hi Tigran,
> Thanks for the questions.
> 
> 
> 
> > On Jul 24, 2018, at 3:37 AM, Mkrtchyan, Tigran <
> > [e-mail]> wrote:
> > Hi Trond,
> > 
> > ----- Original Message -----
> > > From: "Trond Myklebust" <[e-mail]>
> > > To: "Dave Noveck" <[e-mail]>
> > > Cc: "NFSv4" <[e-mail]>, "Thomas Haynes" <
> > > [e-mail]>
> > > Sent: Wednesday, May 30, 2018 5:56:44 PM
> > > Subject: Re: [nfsv4] Review of draft-haynes-nfsv4-delstid-00
> > > Hi Dave,
> > > 
> > > While read delegations may not allow the client to cache byte
> > > range locks,
> > > they do allow it to cache OPENs provided that the application
> > > only asks for
> > > a share lock of OPEN4_SHARE_DENY_NONE or OPEN4_SHARE_DENY_WRITE,
> > > in
> > > combination with OPEN4_SHARE_ACCESS_READ. See for instance:
> > > https://tools.ietf.org/html/rfc5661#section-10.4
> > > 
> > > Concerning your comments around the offline bit. I do agree that
> > > an open
> > > mode requesting that the server fail if the file is offline would
> > > be useful
> > > for a number of cases (e.g. when dealing with backup
> > > applications). One
> > > main use case for the offline bit is to support existing desktop
> > > file
> > > browsers that want to highlight files that are offline to ensure
> > > the user
> > > does not try to open them.
> > 
> > I very much like the idea of offline bit. However, for me, your
> > proposal doesn't go
> > deep enough. First of all, not all offline storage systems have a
> > similar latency.
> > A S3 storage may have couple of seconds, but with a tape archive
> > you can easily get
> > minutes, hours or, in extreme case, even days. Thus, if you are
> > introducing a new attribute,
> > may be instead of binary switch add something like an 'estimated
> > access latency',
> > with zero for online files and other values for offline files:
> > 
> > 
> > const FATTR4_ACCESS_LATENCY            = 83;
> > typedef nfstime4 fattr4_estimated_access_latency;
> > 
> > 
> > The value is a hint, of course.
> 
> I don’t think Linux has the concept of an offline file. Windows does,
> but it only has theboolean value.
> So either the client or Samba server would have to translate this
> range to a binary value?
> I’m not saying no, I’m asking how this would be used.
> Another thing here is that I tend to think not just in the access
> latency, but also in thecost of loading the file. I.e., if it is on-
> site and on a tape archive, I may have a low cost,but high latency to
> load the file. This is versus something in the cloud, which maybe
> high cost but low(er) latency.
> Perhaps my above question about how this would be used really
> reflects that thevalue is subjective.
> > Finally, an open flag OPEN4_ONLINE_ONLY will make life much simpler
> > for end-user applications.
> > A corresponding error like NFS4ERR_OFFLINE or NFS4ERR_DELAY will
> > tell client that it
> > have to explicitly ask for bringing files ONLINE. However, we our
> > old-good friend POSIX
> > will resist, as usual. O_NONBLOCK is not allowed for regular files
> > and EAGAIN is not
> > a valid error code for open call.
> 
> So the client blocks the open call until it succeeds? Yes, I
> understand this may not be allowed.
> 
> > BTW, how you want to propagate offline bit to a file browser?
> 
> I assume it is either an extended attribute or something that gets
> retransmitted over another protocol.
> I.e., in the case we have the Data Portal (translates NFSv3 or SMB to
> NFSv4) will use NFSv4 toget the attribute and then the Samba code
> will send the offline state over SMB to the client.
> 
> > Regards,
> >   Tigran.
> > 
> > 
> > > Concerning the use of bitmaps in open-support, the problem is
> > > that the OPEN
> > > flags are currently an unholy mixture of flags and bit fields (in
> > > the C
> > > language sense). For instance, the OPEN4_SHARE_ACCESS_WANT_* are
> > > mostly a
> > > bit field taking values between 0x0 and 0xFF, and so we require
> > > that
> > > extensions for new share want modes will be adding values
> > > starting at
> > > 0x06... Furthermore, not all servers currently support all the
> > > existing
> > > modes (e.g. OPEN4_SHARE_ACCESS_WANT_CANCEL), so we should require
> > > that
> > > clients be able to detect this.
> > > The same argument is true of the claim type: nobody currently
> > > implements
> > > CLAIM_DELEGATE_PREV or CLAIM_DELEG_PREV_FH, so any future client
> > > that wants
> > > to figure out if the server supports those modes cannot do so.
> > > Assuming
> > > that just because the server implements CLAIM_FH, then it must
> > > implement
> > > all the modes with values < CLAIM_FH is patently false.
> > > 
> > > Concerning your question around the time attribute proxying, the
> > > reason why
> > > we should add new attributes, is because the behaviour of SETATTR
> > > using
> > > these attributes differs from the behaviour when setting mtime
> > > and atime
> > > directly. In particular, the behaviour of the ctime is very
> > > different. In
> > > order to be able to emulate POSIX semantics on the client, we do
> > > not want
> > > the server to change the ctime at all when we set the atime when
> > > returning
> > > the delegation. If we're setting the mtime, we want the server to
> > > update
> > > the ctime if and only if the new mtime is more recent than the
> > > existing
> > > value for ctime.
> > > This behaviour should only be allowed if the client holds the
> > > required
> > > delegation type and uses these attributes. Otherwise, standard
> > > semantics
> > > apply when setting the mtime and atime using the existing
> > > time_access_set/time_modify_set attributes.
> > > 
> > > Cheers
> > >  Trond
> > > On Mon, 21 May 2018 at 13:10, David Noveck <[e-mail]>
> > > wrote:
> > > 
> > > > I have read the document and believe all of the extensions
> > > > suggested in
> > > this document are worthy of serious consideration by the working
> > > group,
> > > but, as we are unsure which of these the working group will
> > > choose to
> > > pusue, I don't think we can be sure of the structure of the
> > > working group
> > > document(s) that will be pursued.  I hope we will get a better
> > > sense of
> > > that at IETF 102.
> > > 
> > > > There are a number of specific issues of presentation that I
> > > > want to call
> > > your attention to.   As these made it rather difficult to
> > > understand the
> > > various individual extensions proposed, I suggest that you take
> > > note of
> > > these in preparing further I-D versions.  Some of these might
> > > also be
> > > useful in preparing your presentation for IETF 102.
> > > 
> > > > There was a lot of uncetainty about features involving
> > > > delegations that
> > > results from confusing text that mentions "delegations" but in
> > > fact only
> > > refers to write delegations.  Note that read delegations do not
> > > give the
> > > client the ability to open the file on the client without
> > > reference to the
> > > server but much of the text assumes that returning any delegation
> > > will make
> > > it unnecessary to return an open stateid.
> > > > It is very confusing that section 2 includes two of your
> > > > extensions
> > > without clearly distinguishing between them.  Adding to the
> > > confusion is
> > > the fact that the motivation of the open support attribute is
> > > dependent on
> > > the added flags bits which appear later.  I suggest you give each
> > > of the
> > > proposed extensions its own section, including a statement of the
> > > motivation for it.  Note that section 4.1 is a good explanation
> > > of the need
> > > for the time-attribute proxying feature but it is hidden and
> > > appears after
> > > the feature is laid out.  The place for the motivation of a
> > > feature is near
> > > the beginning of the description that feature.   After all, you
> > > know why
> > > the feature is needed but you can't expect the reader to have the
> > > same
> > > insight, and it is unrealistic to assume he is going to
> > > assimilate the
> > > protocol details and then correlate them with text explaiing th
> > > motivation
> > > that appears later.
> > > > The material in sections 5 and 5.1 is distracting.   You need
> > > > to look at
> > > RFCs 8275 and 8276 as a model for dealing with XDR extension
> > > issues, rather
> > > than the pNFS RFCs.   I don't see the need for a consolidated XDR
> > > file such
> > > as is appropriate to a pNFS mapping type or a separate licensing
> > > section.
> > > 
> > > > With regard to the individual exensions, I have, in some cases,
> > > > issues
> > > with how you have chosen to do the feature while agreeing that
> > > these cover
> > > important issues that it is worthwhile to address:
> > > 
> > > > With regard to the offline file support, I don't agree with the
> > > assumption that the server will necessarily have to bring the
> > > file online
> > > in order to open it.  Also, requiring him to do a GETATTR in one
> > > compound
> > > and then decide whether to open the file adds to complexity.
> > >  Given that
> > > you are extending open, you should consider bits that indicate
> > > that an
> > > offline file is not to be opened, or that bringing the file
> > > online be
> > > deferred until the first IO operation.  I still think the
> > > attribute is
> > > useful but don't think it should be the only  piece of offline
> > > file support
> > > specified.
> > > > With regard to the open-support attribute. I like the general
> > > > idea.  It
> > > is certainly better than forcing the client to try various bits
> > > and deal
> > > with NOTSUPP errors.  However, I think that basing this on
> > > bitmaps adds
> > > unneded complexity, in that you need separate bit number
> > > definitions for
> > > each existing flag and have the task of adding a bit nmber
> > > whenever you
> > > extend the protocol by adding a flag.  It would be simpler for
> > > this
> > > attribute to parallel the current definition of open and use
> > > flags where
> > > OPEN uses flags.  Also, definition of bit numbers for
> > > combinations of flags
> > > (e.g. _NONE, _BOTH) probably will become unworkable if we add
> > > more bits to
> > > share and deny such as DELETE.
> > > > With regard to the time-attribute-proxying feature, I don't
> > > > understand
> > > why new attributes are needed rather than basing this on  setting
> > > the
> > > attributes that are being proxied.  One option that should be
> > > considered is
> > > extending the union time_how to provide for this case. I also
> > > have problems
> > > with requirements that SETATTR's be done the same in COMPOUND as
> > > the
> > > DELEGRETURN.  I'm not sure why this is needed.  If it is really
> > > necessary
> > > that setting of these attribute needs to be tightly bound to the
> > > delegation
> > > return, an extended delegation return operation should be
> > > considered.   If
> > > you do use SETATTRs, then the stateid would have to be that of
> > > the
> > > delegation.  Given what section 18.30 of RFC5661 currently says
> > > about the
> > > stateid in SETATTR, the very limited discussion of the stateid in
> > > that
> > > section would have to be extended.   The best way to do that
> > > would for the
> > > eventual RFC to include a new section essentially replacing
> > > section 18.30
> > > of RFC5661.
> > > 
> > > 
> > > > _______________________________________________
> > > > nfsv4 mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/nfsv4
> > > 
> > > _______________________________________________
> > > nfsv4 mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/nfsv4