Re: [nfsv4] New draft for working group charter

[David Noveck <[e-mail]>]

> cm-pvt-msg could move forward, but there has been a palpable
> objection to making this work an official standard. Thus I
> don't feel it is ready to promote without further discussion.

I wasn't suggesting that it move forward without further discussion.
I was suggesting that we could well the have necessary discussion
by IETF-99.

My understanding of the objection is to it being a standards-track
document so that it could move forward as a WG Experiental
document and so get a milestone..

I think this is right as a Proposed Standard but it isn't really worth
spending a lot of time arguing about the matter.  The
important facts are that there is an implementation of this
in the Linux client, and that it could help address two important
performance issues in RPC-over-RDMA Version One: invalidation
overhead and excessive use of reply chunks.   As a result, server
implementers that care about performance will implement it,
whether it is a Proposed Standard, an Experimental RFC, an
Informational RFC, a work-in-progress, or an expired individual
submission.

I think it is best for us to have the discussion and move it forward
with whatever status the working group can agree on.

> RPC-over-RDMA Version Two could become a large project.

It doesn't have to be.  As I recall, you proposed, with good
reason, a very small Version Two.

As far as I'm concerned Version Two, as defined in
draft-cel-rpcrdma-version-two
is essentially Version One Done Right.   It is designed so that it is easy
to
create an implementation that interoperates with Version One peers.  The
changes are small and limited to:

   - A larger default inline threshhold.
   - A simple mechanism to exchange information about transport properties
   - The ability to define OPTIONAL extensions,
   - A more flexible remote invalidation scheme

> There are some issues that the WG needs to consider
> carefully before enabling such an effort, most especially
> what is the appetite by vendors/implementers for a new
> version of the protocol, but also, who has the resources
> to construct prototypes?

I think the only way to find out is to ask people and the best way
to do that is to discuss the issue of moving this forward.  If  we had
that discussion and the working group was supportive, I would
probably look, when I implemented a Version One server, how
much work it would take to turn into a Version Two

I think we should just let the working group decide this issue.  if we
can't do that by IETF99, we might be able to do so by the Fall
Bakeathon.  I don't see what kind of consideration you are anticipating
having before proceeding to have the working group discussion.
server.   At that point i would have a better idea of

> Do we understand and agree on
> what problems we are trying to address in the longer term?

No we don't but that is the reason for an extensible protocol.
We need that sort of agreement when we decide on what
extensions to develop.  I agree that that decision is a ways off.

> Have we reached the end of RoRV1's ability to deliver
> performance?

Probably not.  There is a lot of implantation work to be done, but if
people do find bottlenecks that require an extension, it is better to be
prepared.  The reason for Version Two is to be able to make limited
changes when necessary.  It will not make performance improvements
on its own, except perhaps in environments that could benefit from the
more general remote invalidation support.

> There have been questions about the scope of RoRV2 for a
> while.

I recall haring claims that a big Version Two is needed but I
haven't heard them for a while.  If there are still people
advocating a large/massive Version Two, we can have the
necessary discussion as part of proposing advancing our
minimal Versiion Two.

> The answer shifts depending on whether we think
> there is or is not a suitable out-of-band mechanism for
> enabling Remote Invalidation and large inline thresholds.

I don't think that's the case.  I think the question of
the out-of-band mechanism affects the urgency of
proceeding with a minimal Version Two.

Put another way, if cm-pm-msg had never existed, the
Version Two would be an urgent matter to be addressed
immediately.  Given that it does exist, I think we can take
a more leisurely approach and look to make this decision
in the next six months.

> To promote this document I would like to have clarification
> of the scope of RoRV2; and that in turn requires clarity on
> the disposition of cm-pvt-msg.

I don't see the need but I think the clarity is availble.  The formalites
don;t really matter.  What matters is running code and it appears that
the running code provides the mechanism you are loking for, and just
eined an implemented.  I on't see how that affects the scopt Version Two,
however.  what is in rpcrdma-vrsion-two seems a goo fit for our current
need in any case.

> I'm struggling with an existential question.

I'll discuss the issue of performance being explicitly mentioned in the
charter
tomorrow.





On Thu, May 11, 2017 at 4:53 PM, Chuck Lever <[e-mail]> wrote:

>
> > On May 11, 2017, at 2:56 PM, David Noveck <[e-mail]> wrote:
> >
> > > Specific milestones should include:
> >
> > I want to explain that my list could not include a lot of things you
> mention because I felt it best to limit it to WG items.  If there are
> documents that become working group items between now and when the charter
> is submitted, then they should included.
>
> Fair enough. We need some gauge of WG consensus, focus,
> and energy.
>
>
> > I'm not sure we would be able to include things that aren't working
> group items at charter submission, although i believe that we would be able
> to take advantage of the provision that Spencer S. suggested to allow later
> addition of milestones :-)
> >
> > -  Trunking (multi-pathing)
> >
> > This has no current WG item.   I expect there would be some discussion
> of  possible paths forward in this area once migration-issues-13 is
> published.  I'm going to be talking about this area at IETF99 and hope that
> Andy will be able to as well.   I'm hoping that soon after IETF99 we will
> able to add one or more milestones here.
> >
> >  - NVMe layout type
> >  - RDMA layout type
> >
> > For these there is no current WG document or I-d.  I am looking forward
> to work being done on these but don't expect these to be included in the
> charter submission.
> >
> >  - flexfile layout type
> >
> > I did ask Tom privately about setting a target date for this since it is
> a WG item.  I anticipate correcting this at IETF99 by guessing at dates
> until Tom nods his head.
> >
> > > These are either far along, or there are identified authors and energy
> and the work is well-scoped.
> >
> > True but I used having a WG document started or at least agreed upon as
> the gating criterion.  I'm willing to change that if you suggest an
> alternative criterion.
> >
> > > Other work items are less mature.
> >
> > I don't agree. Specifically, I feel the following I-d's should be
> considered for WG status and that could well happen  before, at, or soon
> after IETF-99:
> >       • draft-cel-nfsv4-rpcrdma-version-two
> >       • draft-cel-nfsv4-rpcrdma-cm-pvt-msg
> > If that happens, we could add one or more milestones for those.
>
> cm-pvt-msg could move forward, but there has been a palpable
> objection to making this work an official standard. Thus I
> don't feel it is ready to promote without further discussion.
>
> RPC-over-RDMA Version Two could become a large project.
>
> There are some issues that the WG needs to consider
> carefully before enabling such an effort, most especially
> what is the appetite by vendors/implementers for a new
> version of the protocol, but also, who has the resources
> to construct prototypes? Do we understand and agree on
> what problems we are trying to address in the longer term?
> Have we reached the end of RoRV1's ability to deliver
> performance?
>
> There have been questions about the scope of RoRV2 for a
> while. The answer shifts depending on whether we think
> there is or is not a suitable out-of-band mechanism for
> enabling Remote Invalidation and large inline thresholds.
> To promote this document I would like to have clarification
> of the scope of RoRV2; and that in turn requires clarity on
> the disposition of cm-pvt-msg.
>
>
> > > I don't think the focus of this section should be strictly on RDMA.
> >
> > It isn't.
> >
> > We've introduced performance features in other areas of the stack besides
> > pNFS and RDMA.
> >
> > True.
> >
> > > For example, READ_PLUS, S2SC, and ALLOCATE are intended to
> > > help performance. One of the performance issues we chronically face is
> > t> hat NFS itself is still chatty (eg. Tom's suggestion about reducing
> the
> > > round-trip overhead of OPEN with delegation).
> >
> > I felt that these changes were dealt with adequately in the Extension
> section.  I guess I
> > could rename "Performance Challenges" to be "Major Performance
> Challenges".
> > If this section were to include any performance-related improvements, I
> feel it
> > would lose necessary focus
> >
> > > There is no mention of making NFS into a premier storage technology for
> > > supporting virtualization.
> >
> > No there isn't.  Is there some addition to the charter draft you would
> like to suggest?
>
> I'm struggling with an existential question.
>
> I'm wondering why there is now a special charter focus on
> performance, but not on virtualization, which has been a
> de facto focus of WG effort for the past five years.
> Why isn't it enough to say "The WG is responsible for
> extending these protocols as needed."
>
> If there is a constraint on what may be included in our
> charter, it seems like there will be a problem including
> performance in particular, as there are no WG documents
> specifically about performance.
>
> If protocol innovation is indeed driven by the individuals
> on the WG, then it seems to me that "performance" should
> not be part of our charter. Improving performance should
> be facilitated by the WG / IETF partnership, but driven by
> the diaspora.
>
> I'm comfortable with the Maintenance and Extension sections,
> and might even add -- for completeness -- that the WG is
> responsible for approving changes to certain sections of
> the IANA registries.
>
> I am in philosophical agreement with the technical goal of
> better performance, and will continue working on it. Still
> not sure whether that is something that needs to appear in
> the WG charter.
>
> Spencer (D or S) would you happen to have a preferred BCP
> or RFC that discusses the requirements of a WG charter?
>
>
> > I certainly approve of the goal.  I just don't think it is charter-ready
> yet, but will
> > look carefully at any suggested text you provide.  We need to explain the
> > need/justification for this clearly to Beepy and the Spencers.
> >
> > > RDMA helps there, as do the new features in NFSv4.2.
> >
> > > New security models are needed to properly handle multi-tenancy,
> > > for example. But we've moved ahead in this area for years (NFSv4.2)
> > > without mention in the WG charter.
> >
> > NFSv4.2 is mentioned in the existing charter.  It is not in my charter
> draft,
> > because that work is done, except for post-RFC7862 extensions to v4.2.
> >
> >
> > On Thu, May 11, 2017 at 12:58 PM, Chuck Lever <[e-mail]>
> wrote:
> >
> > > On May 11, 2017, at 7:56 AM, David Noveck <[e-mail]>
> wrote:
> > >
> > > The following is the second iteration of my attempt to provide a draft
> charter to be "whacked" (as Beepy put it) on the working group list so all
> comments are appreciated.  However, if someone would like to present an
> alternate draft, that's OK too.  We do need some sort of draft to work from
> for a discussion in Prague.  Without it, a discussion in Prague is not
> going to arrive at a useful charter candidate.
> > >
> > > This has the following changes from the initial draft.  If you object
> to any of these changes, let me know right away:
> > >       • Deletion of References to FedFS, per Chuck's comments.
> > >       • Inclusion of other ONC components in the Maintenance section,
> per Chuck's comments.
> > >       • Deletion of the out-of-date sections RFC5664bis and NFSv4.2.
> > >       • Deletion (for now) of the section NFSv4 Multi-Domain Access
> for FedFS.  Unlike the previously mentioned sections, this could come back,
> probably in the form NFSv4 Multi-Domain Access if someone (e.g. Andy)
> provides a draft charter section.
> > >       • Added a reference within the Maintenance section, to the
> ability of technical updates to NFSv4 versions to include limited XDR
> changes.
> > >       • A start at a draft Milestones section (see below).  This is
> very limited since most WG documents are past WGLC right now.  I've made
> some reasonable assumptions regarding migration-issues.  When -13 is
> published, the way forward in this area will clearer and the list could be
> updated to include some expected documents to address the issues described
> in the Informational document.  BTW, I'm assuming there will be a need for
> WGLC for that document as a means of establishing consensus on a way
> forward for those issues even though I believe there is no point in
> publishing that document as an RFC.
> > >       • In order to help deal with our  limited set of current
> milestones, I've followed Spencer's suggestion and adapted the material he
> cited from the TCPM charter.
> > > Draft Charter for Working Group (Iteration Two)
> > > NFS Version 4 is the IETF standard for file sharing. To maintain NFS
> Version 4's utility and currency, the working group is chartered to
> maintain the existing NFSv4.0, NFSv4.1, NFSv4.2, protocols and related
> specifications of ONC components such as those defining RPC, XDR, and
> RPCSECGSS. In addition, extensions will be developed, as necessary, to
> correct problems with the protocols as currently specified, to accommodate
> needed file system semantics, and to make significant performance
> improvements.
> > >
> > > Maintenance
> > >
> > > The working group has found that as NFSv4 implementations mature and
> deployments continue, clarifications to existing RFCs are needed. These
> clarifications assist vendors in delivering quality and interoperable
> implementations. The working group is chartered with the vetting of the
> issues and determining correctness of submitted errata. In addition, some
> areas may need more concentrated work to correct the specifications already
> published or to deal with unanticipated interactions between features. In
> the cases in which the required changes are inappropriate for the errata
> system, the working group will assist in publication of best practices RFCs
> or of RFCs that provide editorial modification or technical updates to
> original RFCs.  Once, the new NFSv4 versioning framework is approved, such
> technical updates to NFSv4 versions could include limited XDR changes.
> > >
> > > Extension
> > >
> > > The NFSv4 protocol is designed to allow extension by the addition of
> new operations or new attributes, the creation of minor versions, and the
> definition of new pNFS mapping types.  The working group will discuss
> proposals for such extensions and assure they have adequate technical
> review including discussion of their interaction with existing features
> before adopting them as working group items and helping to draft
> specification documents. Siniilarly, associated ONC protocol components
> that have a versioning/extension  framework can be suitably extended to
> accommodate new security requirements, and to make significant performance
> improvements.
> > >
> > > Performance Challenges
> >
> > Opportunities? Just a thought.
> >
> >
> > > The increase of network bandwidths and the reduction of latencies
> associated with network traffic and access to persistent storage have
> created challenges for remote file access protocols which need to meet
> increasingly demanding performance expectations.  Some work already done in
> this area includes the respecification of RPC-over-RDMA Version One and the
> pNFS SCSI layout.  It is lexpected that further work in this area will be
> required.  This might take the form of further RPC-over-RDMA versions,
> adaptation of the SCSI layout to NVMe, or the development of an
> RDMA-oriented pNFS layout type.  The working group needs to discuss these
> alternatives, and possibly others, and develop the most promising ones.
> >
> > Specific milestones should include:
> >
> >  - Trunking (multi-pathing)
> >  - NVMe layout type
> >  - RDMA layout type
> >  - flexfile layout type
> >
> > These are either far along, or there are identified authors and energy
> > and the work is well-scoped. Other work items are less mature.
> >
> > I don't think the focus of this section should be strictly on RDMA.
> > We've introduced performance features in other areas of the stack besides
> > pNFS and RDMA. For example, READ_PLUS, S2SC, and ALLOCATE are intended to
> > help performance. One of the performance issues we chronically face is
> > that NFS itself is still chatty (eg. Tom's suggestion about reducing the
> > round-trip overhead of OPEN with delegation).
> >
> > There is no mention of making NFS into a premier storage technology for
> > supporting virtualization. RDMA helps there, as do the new features in
> > NFSv4.2. New security models are needed to properly handle multi-tenancy,
> > for example. But we've moved ahead in this area for years (NFSv4.2)
> > without mention in the WG charter.
> >
> >
> > > Milestones (Preliminary draft)
> > >
> > > Because the previous charter was at variance with the work the group
> was actually doing, the list of pending milestones that can determined now
> is quite limited.  To accommodate this situation and in light of the fact
> that maintenance activities are inherently unpredictable, new milestones
> that fall within the scope specified within the charter can be added after
> working group consensus on acceptance and approval by the responsible Area
> Director.
> > >
> > > Date               Milestone
> > >
> > > Nov. 2017      Publication of RFC5666bis as a Proposed Standard
> > >
> > > Nov. 2017      Publication of a Proposed Standard describing
> bidirectional operation for RPC-over-RDMA
> > >
> > > Jan. 2018       Publication of RFC5667bis as a Proposed Standard.
> > >
> > > Feb. 2018       Publication of a Proposed Standard describing NFSv4
> versioning/extension framework.
> > >
> > > Feb. 2018       Publication of a Proposed Standard describing the
> umask extension to NFSv4.2.
> > >
> > > Feb. 2018       Publication of a Proposed Standard describing the
> xattr extension to NFSv4.2.
> > >
> > > March 2018    WGLC for draft-ietf-nfsv4-migration-issues
> (Informational)
> > >
> > >
> > >
> > > _______________________________________________
> > > nfsv4 mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> > --
> > Chuck Lever
> >
> >
> >
> >
>
> --
> Chuck Lever
>
>
>
>