Re: [nfsv4] Review of draft-ietf-nfsv4-rfc5667bis-09

[Chuck Lever <[e-mail]>]

> On May 8, 2017, at 2:53 PM, Tom Talpey <[e-mail]> wrote:
> 
> On 5/8/2017 1:34 PM, Chuck Lever wrote:
>> 
>>> On May 8, 2017, at 9:58 AM, Chuck Lever <[e-mail]> wrote:
>>> 
>>>> 
>>>> On May 8, 2017, at 8:35 AM, Tom Talpey <[e-mail]> wrote:
>>>> 
>>>> On 5/4/2017 11:00 AM, Chuck Lever wrote:
>>>>> I've tried to address your concerns, and I changed the segment count
>>>>> limit to document what we know about the current Solaris implementations.
>>>>> I liked the "be conservative in what you send" idea, but I couldn't find
>>>>> a way to work it in that was not awkward.
>>>>> 
>>>>> If this text is OK, I'll submit a fresh I-D revision after WGLC ends on
>>>>> Friday.
>>>> 
>>>> Well that wasn't enough time for me to respond since I was traveling
>>>> from SambaXP in Germany, but the text does look better. The statement
>>>> below makes no requirement on the server.
>>>> 
>>>>> 5.4.2.  Chunk List Complexity
>>>>> ...
>>>>> To avoid encountering server chunk list complexity limits, NFS
>>>>> version 4 clients SHOULD follow the below prescriptions when
>>>>> constructing transport headers:
>>>>> 
>>>>> o  The Read list can contain either a Position-Zero Read chunk, one
>>>>>    Read chunk with a non-zero Position, or both.
>>>>> 
>>>>> o  The Write list can contain no more than one Write chunk.
>>>>> 
>>>>> o  Any chunk can contain up to sixteen RDMA segments.
>>>> 
>>>> I would suggest the following modifications to the above.
>>>> 
>>>> "In accordance with chunk list processing limits of existing
>>>> server implementations, NFS Version 4 clients SHOULD maximally
>>>> follow the below prescriptions when constructing requests to
>>>> be transported by RPC-over-RDMA. Additionally, NFS Version 4
>>>> servers SHOULD minimally be prepared to accept and process such
>>>> requests."
>>> 
>>> Actually I think "servers MUST be prepared" is appropriate here,
>>> although IMO "clients should follow the below" implies the server
>>> side requirements already. But no harm in making it explicit.
>>> 
>>> 
>>>>> NFS version 4 clients wishing to send more complex chunk lists can
>>>>> provide configuration interfaces to bound the complexity of NFS
>>>>> version 4 COMPOUNDs, limit the number of elements in scatter-gather
>>>>> operations, and avoid other sources of RPC-over-RDMA chunk overruns
>>>>> at the receiving peer.
>>>>> 
>>>>> An NFS Version 4 server has some flexibility in how it indicates that
>>>>> an RPC-over-RDMA Version One message received from an NFS Version 4
>>>>> client is valid but cannot be processed.  Examples include:
>>>> 
>>>> 
>>>> Strongly suggest deleting the words "has some flexibility" and "Examples
>>>> include". We've already had this discussion, so I will just suggest the
>>>> following.
>>>> 
>>>> "An NFS Version 4 server SHOULD return one of the following responses
>>>> to a client which has sent a request containing RPC-over-RDMA chunks
>>>> which cannot be processed due to exceeding server implementation
>>>> limits"
>>> 
>>> Both look fine to me. Spencer, if you're willing, I can replace -10
>>> with a revision that includes these changes.
>> 
>> Here's the new text. I'll wait for Tom to respond before cutting -11.
>> 
>> 
>> 5.4.2.  Chunk List Complexity
>> 
>>   The RPC-over-RDMA Version One protocol does not place any limit on
>>   the number of chunks or segments that may appear in Read or Write
>>   lists.  However, for various reasons NFS version 4 server
>>   implementations often have practical limits on the number of chunks
>>   or segments they are prepared to process in a single RPC transaction
>>   conveyed via RPC-over-RDMA Version One.
>> 
>>   These implementation limits are especially important when Kerberos
>>   integrity or privacy is in use [RFC7861].  GSS services increase the
>>   size of credential material in RPC headers, potentially requiring
>>   more frequent use of Long messages.  This can increase the complexity
>>   of chunk lists independent of the NFS version 4 COMPOUND being
>>   conveyed.
>> 
>>   In accordance with chunk list processing limits of existing server
>>   implementations, NFS Version 4 clients SHOULD follow the
>>   prescriptions listed below when constructing RPC-over-RDMA Version
>>   One messages.  NFS Version 4 servers MUST be prepared to accept and
>>   process such requests.
> 
> My only comment is on the above paragraph, on which I had suggested
> adding "maximally" and "minimally" in an attempt to convey that these
> were not absolute requirements, i.e. no more, no less than the listed
> values. That's not the intent, right?

The server is required to accept the listed bullets at a minimum,
and the client should not send more than the list bullets, but
it can send more if it is confident the server supports it. And,
the stated limits are based on existing implementations, not on
any particular limit in the transport protocol.

I think we agree on the intent.


> Again, my suggestion is to state these as "safe" values, to be used by
> the client when no other knowledge of the actual limits exists.

I had some trouble with "safe" because it's kind of an undefined
term in this context. My preference is the paragraph should state
what it means directly.

It still feels to me like "SHOULD follow" means it is OK for the
client to exceed the stated limits. So the current text says the
same thing with and without "maximally" or "minimally" and already
conveys the above intent.


> The server text is ok.
> 
> Along those lines then, how about:
> 
>  "In accordance with chunk list processing limits of existing server
>   implementations, in the absence of explicit knowedge of the server's
>   limits, NFS Version 4 clients SHOULD follow the prescriptions listed
>   below when constructing RPC-over-RDMA Version One messages.  All NFS
>   Version 4 servers MUST be prepared to accept and process such
>   requests."

My counter-offer ;-)

   In the absence of explicit knowledge of the server’s limits, NFS
   Version 4 clients SHOULD follow the prescriptions listed below when
   constructing RPC-over-RDMA Version One messages.  NFS Version 4
   servers MUST accept and process such requests.


> Tom.
> 
> 
>>   o  The Read list can contain either a Position-Zero Read chunk, one
>>      Read chunk with a non-zero Position, or both.
>> 
>>   o  The Write list can contain no more than one Write chunk.
>> 
>>   o  Any chunk can contain up to sixteen RDMA segments.
>> 
>>   NFS version 4 clients wishing to send more complex chunk lists can
>>   provide configuration interfaces to bound the complexity of NFS
>>   version 4 COMPOUNDs, limit the number of elements in scatter-gather
>>   operations, and avoid other sources of chunk overruns at the
>>   receiving peer.
>> 
>>   An NFS Version 4 server SHOULD return one of the following responses
>>   to a client that has sent an RPC transaction via RPC-over-RDMA
>>   Version One which cannot be processed due to chunk list complexity
>>   limits on the server:
>> 
>>   o  A problem is detected by the transport layer while parsing the
>>      transport header in an RPC Call message.  The server responds with
>>      an RDMA_ERROR message with the err field set to ERR_CHUNK.
>> 
>>   o  A problem is detected during XDR decoding of the RPC Call message
>>      while the RPC layer reassembles the call's XDR stream.  The server
>>      responds with an RPC reply with its "reply_stat" field set to
>>      MSG_ACCEPTED and its "accept_stat" field set to GARBAGE_ARGS.
>> 
>>   After receiving one of these errors, an NFS version 4 client SHOULD
>>   NOT retransmit the failing request, as the result would be the same
>>   error.  It SHOULD immediately terminate the RPC transaction
>>   associated with the XID in the reply.
>> 
>> 
>>>> Tom.
>>>> 
>>>> 
>>>>> o  A problem is detected by the transport layer while parsing the
>>>>>    transport header in an RPC Call message.  The server responds with
>>>>>    an RDMA_ERROR message with the err field set to ERR_CHUNK.
>>>>> 
>>>>> o  A problem is detected during XDR decoding of the RPC Call message
>>>>>    while the RPC layer reassembles the call's XDR stream.  The server
>>>>>    responds with an RPC reply with its "reply_stat" field set to
>>>>>    MSG_ACCEPTED and its "accept_stat" field set to GARBAGE_ARGS.
>>>>> 
>>>>> After receiving one of these errors, an NFS version 4 client SHOULD
>>>>> NOT retransmit the failing request, as the result would be the same
>>>>> error.  It SHOULD immediately terminate the RPC transaction
>>>>> associated with the XID in the reply.
>>>> 
>>>> 
>>>> _______________________________________________
>>>> nfsv4 mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/nfsv4
>>> 
>>> --
>>> Chuck Lever
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> nfsv4 mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/nfsv4
>> 
>> --
>> Chuck Lever
>> 
>> 
>> 
>> _______________________________________________
>> nfsv4 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nfsv4
>> 
>> 
> 
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4

--
Chuck Lever