Re: [nfsv4] Fwd: I-D Action: draft-ietf-nfsv4-rfc5667bis-06.txt

[Chuck Lever <[e-mail]>]

> On Feb 24, 2017, at 2:32 PM, David Noveck <[e-mail]> wrote:
> 
> Let's start with the most controversial area and try to put that aside for the moment.  Chuck and I continue to have different opinions about the appropriate way for ULBs (all of which that exist are in the current document) to deal with the fact that, NFS-over-RDMA is no longer solely represented by Version One.  Chuck and I have had extensive discussions and decided that we could not resolve this in a reasonable time.   As a result, I accepted Chuck's decision to proceed with rfc5667bis on a Version-One-only basis and agreed to review -06 on that basis.
> 
> Others in the working group are free to take a different view but I feel that it would not be helpful to delay rfc5666bis because of this issue.   The working group reached consensus on these documents about a decade ago and only now are we reaching an equivalent state with the revived documents.  Further transport versions are needed to get us to the protocol we want, but the working group has put a priority on reviving Version One and I would urge people not to add unnecessarily to the delay in getting that part of our work done.
> 
> I will be submitting a draft-dnoveck-nfsv4-nfsulb that illustrates how a more transport-generic NFS ULB might be written.  Chuck and I have repeatedly disagreed about how difficult (or even possible) it was to do this.  It is hard to discuss this issue in the abstract so that once we have a document to look at and review, the working group can see for itself whether this is a viable approach and decide if it might be more appropriate in the long term, as the transport is extended.

The function of a ULB is to adapt the use of specific
transport mechanisms to an Upper Layer Protocol. It's a
practical document written for implementers.

There is more to a ULB than noting which XDR data items
are DDP-eligible.

The current NFS ULB has several sections that account for
deficiencies in RPC-over-RDMA Version One. If the WG designs
subsequent transport protocols correctly, future ULBs will
likely not need to address some of these deficiencies, or
could need to address them in a different way.

Future ULBs might need new discussion to address mechanisms
that we can't now predict. We have right now only one extant
protocol version, one extant ULB, and one direct data
placement mechanism. There are some personal drafts that
discuss possible futures, but IMO the WG can't build a
Standards Track document on those ideas until they
themselves are advanced to RFC documents.

So I ask: Based on the flawed ULP and transport we have
today and this set of speculative proposals, how will the
WG provide a guarantee of a lasting and high quality
agnostic ULB?

There are a long list of tasks ahead of us, including
implementing prototypes and productizing new protocols. Is
an agnostic ULB going to benefit implementers directly, or
are implementers not the target audience of such a document?

I'd rather keep extension and ULB documents small and narrow
in focus. That way they are easier to author, review, and
implement, and the ULBs can discuss transport and ULP issues
with appropriately precise language about specific well-
documented protocols.

If the WG creates an agnostic ULB, it makes sense as an
informational document that feeds the process of protocol
design. In that case, let's not call it a ULB at all.


> This document is based on rfc5667bis so that Chuck would normally be listed as a co-author, but Chuck has decided against that.  I hope he will reconsider.
> 
> 
> 
> On Fri, Feb 24, 2017 at 12:00 PM, Chuck Lever <[e-mail]> wrote:
> 
> 
> > Begin forwarded message:
> >
> > From: [e-mail]
> > Subject: [nfsv4] I-D Action: draft-ietf-nfsv4-rfc5667bis-06.txt
> > Date: February 24, 2017 at 8:50:16 AM PST
> > To: <[e-mail]>
> > Cc: [e-mail]
> >
> >
> > A New Internet-Draft is available from the on-line Internet-Drafts directories.
> > This draft is a work item of the Network File System Version 4 of the IETF.
> >
> >        Title           : Network File System (NFS) Upper Layer Binding To RPC-Over-RDMA Version One
> >        Author          : Charles Lever
> >       Filename        : draft-ietf-nfsv4-rfc5667bis-06.txt
> >       Pages           : 19
> >       Date            : 2017-02-24
> >
> > Abstract:
> >   This document specifies Upper Layer Bindings of Network File System
> >   (NFS) protocol versions to RPC-over-RDMA Version One.  Upper Layer
> >   Bindings are required in order to enable RPC-based protocols such as
> >   NFS to use Direct Data Placement on RPC-over-RDMA Version One.  This
> >   document obsoletes RFC 5667.
> >
> >
> >
> > The IETF datatracker status page for this draft is:
> > https://datatracker.ietf.org/doc/draft-ietf-nfsv4-rfc5667bis/
> >
> > There's also a htmlized version available at:
> > https://tools.ietf.org/html/draft-ietf-nfsv4-rfc5667bis-06
> >
> > A diff from the previous version is available at:
> > https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-rfc5667bis-06
> >
> >
> > Please note that it may take a couple of minutes from the time of submission
> > until the htmlized version and diff are available at tools.ietf.org.
> >
> > Internet-Drafts are also available by anonymous FTP at:
> > ftp://ftp.ietf.org/internet-drafts/
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/nfsv4
> 
> 
> Changes since -05:
> 
> Dave Noveck observed that my stated goal was "some
> degree of transport version agnosticism" but there
> is still inconsistent language throughout the document.
> I've been struggling with how to discuss certain
> mechanisms, such as short Reply chunk retry, without
> the mention of version-specific transport protocol
> elements.
> 
> Thus I've decided to reverse course and explicitly
> state this document applies to RPC-over-RDMA Version
> One, though not explicitly exclude the use of the ULB
> by future versions or extensions.
> 
>   o A ULB sometimes has to discuss in detail how a ULP
>     might use a transport. For example, it might need
>     to refer to a specific error code or transport
>     mechanism that is available only in some particular
>     transport versions. That's challenging to do when
>     using only abstract terms.
> 
>   o It is not possible to predict how future transports
>     might innovate. Specificity in the text of a ULB can
>     make it difficult to apply easily to later versions
>     of a transport, or its extensions.
>        + For instance, there are a couple places (like
>          dealing with READ_PLUS replies, or how to handle
>          multiple Write chunks) where there is a very
>          transport-specific mechanism described. We could
>          have some innovation in the future to help deal
>          with these situations.
> 
>    o The change in direction resolves some subtle issues
>      about how cross-citation will work when defining a
>      transport version or extension and possibly a ULB to
>      go with it. It appears more straightforward to me to
>      provide a basic ULB that future transport and
>      extensions can point to and say either:
>        + That ULB rule can be re-used, or
>        + That ULB rule can be re-used with these changes
>          or exceptions, or
>        + This is the new ULB rule that must be used for
>          this new mechanism
> 
> This might be a controversial change, but I feel the
> document is more clear now while still enabling its use
> for future transports and their extensions.
> 
> Karen Deitke had questions about whether chunk reduction
> can be used with Long messages. The recent removal of "MUST
> ignore subsequent chunks" seems to have opened a little
> gap. I've added some language which I hope closes that gap.
> 
> While working with IANA on rfc5666bis, Tom Talpey noticed
> that the official IANA registration of the nfsrdma ports
> references RFC 5666, but should reference the NFS ULB. RFC
> 5667 doesn't have an IANA Considerations section, and the
> mention of nfsrdma in RFC 5666 is as an example. Thus the
> IANA registry citation needs to be corrected, and one of
> these documents needs to have an official specification
> of nfsrdma. To make the IANA registry and the updated
> documents consistent, I've updated the IANA Considerations
> sections in both rfc5666bis and rfc5667bis.
> 
> Karen and Dave both suggested a reorganization of the
> discussion of short Reply chunk retry.
> 
> A very recent discussion on [e-mail]
> brought up the question of whether the language of RFC
> 5661 Section 2.9.1 forecloses on the use of RPC-over-RDMA
> with NFS version 4, because of version 4's tighter
> requirements for transport-level congestion avoidance.
> Of special concern was RoCEv2, which is implemented over
> UDP on IP. A section has been introduced to discuss this,
> and summarize the Working Group's (hopefully) approval of
> the use of RPC-over-RDMA for NFS version 4 deployments.
> 
> 
> --
> Chuck Lever
> 
> 
> 
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4
> 

--
Chuck Lever