[nfsv4] Suggested clarifications related to directory delegations

[Rick Macklem <[e-mail]>]

Hi,

Jeff and I have been doing some experimental implementation of
directory delegations.  We found that the text in RFC8881 was
unclear and almost seemed contradictory for directory delegations
and the associated CB_NOTIFYs. There was no clear indication
of when a directory delegation was required to be recalled, for example.

We also have observed that generating some of the position
information (where in a directory the entry is being added/removed)
is onerous and often not needed by clients.  As such, the suggested
text defines three options that a server can follow when filling in
position information in CB_NOTIFYs.

Anyhow, here is some possible text to replace parts of Sec. 10.9.2
and Sec. 20.4.3:

Here is my first stab at a replacement for Sec. 10.9.2:

   NFSv4.1 introduces the GET_DIR_DELEGATION (Section 18.39) operation
   to allow the client to ask for a directory delegation.  The
   delegation covers directory attributes and all entries in the
   directory.
   In addition to asking for delegations, a client can also ask for
   notifications for certain events.  These events include changes to
   the directory's attributes and/or its contents.  If a client asks for
   notification for a certain event, the server will notify the client
   when that event occurs.  The notifications are asynchronous.

   In the NOTIFY4_ADD_ENTRY, NOTIFY4_REMOVE_ENTRY and NOTIFY4_RENAME_ENTRY
   notifications, there is position information.
   Position information, indicating where in the directory the
   entry is being added/removed may be sent to the client in one of three ways.

   (A) - Full position information, as follows:
   If this entry is added to the end of the directory, the
   server will set the nad_last_entry flag to TRUE.  If the file is
   added such that there is at least one entry before it, the server
   will also return the previous entry information (nad_prev_entry, a
   variable-length array of up to one element.  If the array is of
   zero length, there is no previous entry), along with its cookie.

   (B) - Position information for a server that always returns monotonically
   increasing values for directory offset cookies.  Since the new cookie
   defines the new entry's position, the position information may be
   sent as follows:
   A nad_new_entry_cookie of length 1 with the new cookie in it.
   A nad_prev_entry of length 1 with an invalid notify_entry4 indicated
   by the ne_file component of length 0.
   The nad_last_entry can be any value and should be ignored.

   (C) - No position information, as follows:
   A nad_new_entry_cookie of length 0.
   A nad_prev_entry of length 1 with an invalid notify_entry4 indicated
   by the ne_file component of length 0.
   The nad_last_entry can be any value and should be ignored.
   For this option, if nad_old_entry is of length 1, the
   nrm_old_entry_cookie field in it will always be set to 0.

   The server MUST use the same position information option for
   all notifications sent for a given directory.

   If the gddr_notification bitmap in the GET_DIR_DELEGATION reply
   has no bits set other than possibly NOTIFY4_CHANGE_CHILD_ATTRS,
   this is referred to as the pure recall model.
   For the pure recall model, the delegation must be recalled
   synchronously by the server whenever the directory is changed
   via an operation performed by another client.
   The client may indicate that it expects the pure recall model
   by not setting any bit in gdda_notification_types in the
   GET_DIR_DELEGATION request except, optionally, the bit for
   NOTIFY4_CHANGE_CHILD_ATTRS.
   The operation causing the recall will have
   to wait until the recall is complete.  Any changes to directory
   entry attributes will not cause the delegation to be recalled.

   When a client acquires a directory delegation and any bit for a
   notification other than NOTIFY4_CHANGE_CHILD_ATTRS is set in
   gddr_notifications in the GET_DIR_DELEGATION reply, the server
   must recall the delegation if it cannot send a notify
   of the type(s) with bits set in gddr_notification for the delegation.
   This includes the client that caused the directory
   change via an operation unless the server is using position
   information option (C) above.
   For servers using option (C) for position information, the recall
   is required if the operation that changed the directory is performed
   by another client.  Since notifications are asynchronous,
   the recall may be done asynchronously.

   The notifications
   provide a way of avoiding recalls in situations where a directory is
   changing enough that the pure recall model may not be effective while
   trying to allow the client to get substantial benefit.  In the
   absence of notifications, once the delegation is recalled the client
   has to refresh its directory cache; this might not be very efficient
   for very large directories.

   The delegation is read-only and the client may not make changes to
   the directory other than by performing NFSv4.1 operations that modify
   the directory or the associated file attributes so that the server
   has knowledge of these changes.  In order to keep the client's
   namespace synchronized with that of the server, the server will
   notify the delegation-holding client(s) (assuming it has requested
   notifications) of the changes made as a result of that client's
   directory-modifying operations.  This is to avoid any need for that
   client to send subsequent GETATTR or READDIR operations to the
   server.

This is a first stab at replacement text for Sec. 20.4.3, starting at
the 3rd para.:

   If the client holding the delegation makes any changes in the
   directory that cause files or sub-directories to be added or removed,
   the server will notify that client of the resulting change(s) as
   indicated by the bits set in gddr_notification in the
   GET_DIR_DELEGATION reply.
   If the client holding the delegation is making attribute or cookie
   verifier changes only, the server does not need to send notifications
   to that client.

   The server will send the following information for
   each operation:

   NOTIFY4_ADD_ENTRY
      The server will send information about the new directory entry
      being created, optionally along with the cookie for that entry.
      The entry information (data type notify_add4) includes the component name
      of the entry and attributes.  The server will send this type of entry
      when a file is actually being created, when an entry is being
      added to a directory as a result of a rename across directories
      (see below), and when a hard link is being created to an existing
      file.
      If the new entry's cookie is available, it will be in the
      nad_new_entry_cookie (another variable-length array of up to one
      element) field.  This MUST be sent for position information option (B).
      (See Sec. 10.9.2)
      If the addition of the entry causes another entry
      to be deleted (which can only happen in the rename case)
      atomically with the addition, then information on this entry is
      reported in nad_old_entry.

   NOTIFY4_REMOVE_ENTRY
      The server will send information about the directory entry being
      deleted.  The server will also send the cookie value for the
      deleted entry so that clients can get to the cached information
      for this entry.  For position information option (C), the cookie
      value nrm_old_entry_cookie will always be 0.

   NOTIFY4_RENAME_ENTRY
      The server will send information about both the old entry and the
      new entry.  This includes the name and attributes for each entry.
      In addition, if the rename causes the deletion of an entry (i.e.,
      the case of a file renamed over), then this is reported in
      nrn_new_entry.nad_old_entry.  This notification is only sent
      if both entries are in the same directory.  If the rename is
      across directories, the server will send a remove notification to
      one directory and an add notification to the other directory,
      assuming both have a directory delegation.
      The position information sent is as defined for NOTIFY3_ADD_ENTRY
      and NOTIFY4_REMOVE_ENTRY, given the option chosen by the server.

   If the server sets NOTIFY4_ADD_ENTRY and NOTIFY4_REMOVE_ENTRY, but
   not NOTIFY4_RENAME_ENTRY in gddr_notification in the
   GET_DIR_DELEGATION reply, rename operations will generate
   NOTIFY4_ADD_ENTRY and NOTIFY4_REMOVE_ENTRY notifications for renames.

I do not know if these changes are too extensive for RFC5661bis or not?

Any suggestions/changes are welcome, rick