Re: [Gen-art] Gen-art LC review of draft-ietf-nfsv4-rfc3530bis-33

[Elwyn Davies <[e-mail]>]

Hi, David.

Thanks for the response.

A couple of comments below...

Regards,
Elwyn

On 01/12/14 18:50, David Noveck wrote:
>     My view (remains) that if the cookie is not tied to the change
>     attribute of the directory, this makes life difficult for both the
>     server (what does it send if the data has been changed and how does
>     it manage maintaining checkpoints -- where a readdir finished --, in
>     particular if the checkpoint was on an entry that has been deleted),
>     and the client.
>
>
> The cookie is not tied to the change attribute of the directory.
>
> The purpose of the cookie is ensuring that you do a READDIR that
> encompasses multiple requests:
>
>   * entries that exist during the entire period of the multi-requuest
>     directory read appear exactly once in the result.
>   * entries that exists during some part of the period of the
>     multi-request directory read  appear at most once in the result.
>
>
> Basically implementations scan through the directory as it exists on
> disk and if there is a created entry, the client may or may not see it,
> depending on where the the new entry happens to be put, with respect to
> the ongoing scan.  As far as deleted entries, the implementation only
> has to be able to move on to the next entry if it encouneters an entry
> marked as deleted.

OK, so the server has to be clever rather than telling the client to 
restart the readdir -- I'm not sure this is the right trade-off since 
the client has to cope with being told the cached cookie is useless (as 
per Trond's input below) but I guess implementations work this way 
already and it isn't my call.  In that case I suggest that the last 
sentence of s15.26.4, para 9, is modified to tell server implementers a 
bit more clearly what they have to do.  Suggestion:
OLD:
    Ideally, the cookie value should not change if the directory is
    modified since the client may be caching these values.
NEW:
    The server SHOULD try to accept cookie values issued with READDIR
    responses even if the directory has been modified between the
    READDIR calls but MAY return NFS4ERR_NOT_VALID if this is not
    possible as might be the case if the server has rebooted in the
    interim.

One nasty thought occurs to me regarding the idea that a client caches
the cookie value(s):  Is the server supposed to maintain the whole
sequence of cookies for a sequence of READDIRs on a single directory so
the client can reissue some of the READDIR calls or is it fair enough 
for the server to throw them away after the client has used them once? 
And how long should the server hang onto cookie state in normal operation?

>
>     s19.4.1, para 2:
>
>     >    For the purposes of open delegation, READs and WRITEs done without an
>     >    OPEN are treated as the functional equivalents of a corresponding
>     >    type of OPEN.  This refers to the READs and WRITEs that use the
>     >    anonymous stateid.  Therefore, READs or WRITEs with an anonymous
>     >    stateid done by another client will force the server to recall a
>     >    OPEN_DELEGATE_WRITE delegation.  A WRITE with an anonymous stateid
>     >    done by another client will force a recall of OPEN_DELEGATE_READ
>     >    delegations.
>
>
> suggest rewriting this as;
>
>     For the purposes of open delegation, READs and WRITEs done without an
>
>     OPEN (anonymous and read-bypass stateids) are treated as the functional
>     equivalents of a corresponding type of OPEN. READs and WRITEs done
>     with an
>     anonymous stateid done by another client will force the server to
>     recall a
>
>     OPEN_DELEGATE_WRITE delegation.  A WRITE with an anonymous stateid
>     done by another client will force a recall of OPEN_DELEGATE_READ
>
>       delegations.  Handling of read-bypass stateid's is identical
>     except that a READ
>
>     done with a read-bypass stateids will not force a recall of an
>     OPEN_DELEGATE_READ
>
>     delegation.
>
That sounds good.  It might be useful to add (e.g., to the end of para 1 
of s9.6.1:
"referred to as anonymous stateid and read-bypass stateid respectively."

>
>
>
> On Mon, Dec 1, 2014 at 9:41 AM, Elwyn Davies <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     Hi, Tom and David.
>
>     Sorry to have been dilatory in running through the v35 that Tom sent me.
>
>     This looks pretty good to me and has sorted out almost everything
>     that I flagged up in the review. In particular s7.8 is much improved.
>
>     The one problem that I have left is the issue of directory contents
>     being changed under the feet of a multi-part readdir request.  The
>     last mail on this subject from 14 November said:
>
>
>
>             On Nov 14, 2014, at 3:05 PM, Tom Haynes
>             <[e-mail]
>             <mailto:[e-mail]>__> wrote:
>
>
>     <<snip>>
>
>                         On Nov 11, 2014, at 12:48 PM, Elwyn Davies
>                         <[e-mail]
>                         <mailto:[e-mail]>
>                         <mailto:[e-mail]
>                         <mailto:[e-mail]>>__> wrote:
>
>
>
>
>                         s15.26.4/s15.26.5: What happens if the directory
>                         has been modified
>                         (entries added or deleted) between calls of
>                         READDIR intended to
>                         retrieve segments of the directory information?
>                         Related to this,
>                         presumably no ordering of entries is implied in
>                         the returned
>                         information - it would be desirable to state this.
>
>
>
>                     This is where the cookieverf is supposed to come in.
>
>                     In most implementations, the cookie needs to be
>                     persistent, which indeed
>                     means an ordering of the information.
>
>
>                 AFAICS the text doesn't tie the cookieverf to the change
>                 attribute of the
>
>     >>> directory.  Presumably this is what is intended - it would certainly mean
>
>     that changes would be picked up and READDIR would have to start from
>     the beginning again.
>
>
>
>             Asking an expert…
>
> 15.26.4
>          From Trond:
>
>         As far as I know, the cookieverf is about validating the cookies
>         themselves. There is no requirement in any of the descriptions in
>         RFC1813, RFC3530 or RFC5661 that states that the contents of the
>         directory need to change when the cookieverf does or vice-versa.
>         In theory, the server could throw out the cookies and regenerate
>         them
>         on reboot, for instance.
>
>
>     My view (remains) that if the cookie is not tied to the change
>     attribute of the directory, this makes life difficult for both the
>     server (what does it send if the data has been changed and how does
>     it manage maintaining checkpoints -- where a readdir finished --, in
>     particular if the checkpoint was on an entry that has been deleted),
>     and the client.
>
>     Thoughts?
>
>
>     I also found a some minor typos (mostly finger trouble in the new
>     bits) and there are a couple of nits outstanding (see below).
>
>     Are you intending to publish a -35 before Thursday?
>
>     Regards,
>     Elwyn
>
>     Nits:
>     =====
>     s2.1: nfs_lease4 definition still missing.
>
>     s8.5, last para: I am not sure that the jargon 'trunking relation'
>     is widely understood. [I don't think I have ever come across it.]
>
>     s19.4.1, para 2:
>      >    For the purposes of open delegation, READs and WRITEs done
>     without an
>      >    OPEN are treated as the functional equivalents of a corresponding
>      >    type of OPEN.  This refers to the READs and WRITEs that use the
>      >    anonymous stateid.  Therefore, READs or WRITEs with an anonymous
>      >    stateid done by another client will force the server to recall a
>      >    OPEN_DELEGATE_WRITE delegation.  A WRITE with an anonymous stateid
>      >    done by another client will force a recall of OPEN_DELEGATE_READ
>      >    delegations.
>
>     Did you mean to take out the 'all 1's case' which was an alternative
>     in v33?
>
>     s9.10, para 1:
>
>         to use a stateid of all 0's or all 1's
>
>     Ought to be changed to "anonymous" and "READ bypass"
>
>     s9.11, last para; s15.18.5, 5th from last para; s15.20.4, para 1:
>     Reference s9.1.3 about incrementing seqid. (Three places all told.)
>
>     Editorial:
>     ==========
>
>     s5.4, 2nd bullet labelled '1.': s/common to all object within given
>     file system./common to all objects within the given file system./
>     [Very nitty: Could use a different label type for the second set of
>     bullets.]
>
>     s6.2: s/RECOMMENED/RECOMMENDED/
>
>     s8.8, para 1: s/an zero-length string/a zero-length string/ [Missed
>     it last time].
>
>     s9: s/has a secondary purpose/have a secondary purpose/
>
>     New s9.1.3 looks good -- except for a bit of finger trouble
>     s9.1.3, para 4: s/cuurent/current/
>     s9.1.3, para 5: s/raesonably/reasonably/
>     s9.1.3, para 6: s/wraparoundby/wraparound by/, s/gretater/greater/
>
>     s9.1.6, para 2: s/consiting/consisting/
>
>     s9.1.7, para 1: s/Subseuent/Subsequent/
>
>     s10.3.1, first bullet: s/(such as an OPENs specifying
>     DENY=NONE)/(such as for [or at or on] OPENs specifying DENY=NONE)/
>
>     s12.4, 1st bullet: s/has a non-UT8 encoded/has a non-UTF8 encoded/
>                                     ^^^
>     s15.2.5, para 2: s/in which operation that/in which operations that/
>
>     s15.11.5, para 5: s/the server may return the error
>     NFS4ERR_NOTSUPP./the server MAY return the error NFS4ERR_NOTSUPP./
>     (I think)
>
>     s15.18.6, last para: s/tfor the same client/for the same client/,
>                  s/i size and change attribute/size and change attributes/
>                    ^^^                     ^^^
>
>     s15.26.4/s15.26.5:  Not convinced about discussion on directory info
>     changing under the feet of readdir.
>
>     s17, para 7: s/match wth the previous/match with the previous/
>                           ^^^
>
>
>
>
>     On 18/11/14 15:17, Tom Haynes wrote:
>
>
>             On Nov 18, 2014, at 1:20 AM, Elwyn Davies
>             <[e-mail] <mailto:[e-mail]>> wrote:
>
>             Hi.
>
>             Could you send me a snapshot of the updated version?  I am
>             getting a bit lost on the results of the various changes.
>
>             Cheers,
>             Elwyn
>
>
>
>
>         Hi,
>
>         I was wondering about that. :-)
>
>         Later,
>         Tom
>
>
>
>
>
>
>