Re: [nfsv4] rfc5667bis open issues

[Chuck Lever <[e-mail]>]

> On Sep 26, 2016, at 1:20 PM, Chuck Lever <[e-mail]> wrote:
> 
>> 
>> On Sep 24, 2016, at 8:11 AM, David Noveck <[e-mail]> wrote:
>> 
>>> The issue is that the language allows ERR_CHUNK to be
>>> returned after a server has processed the RPC request when 
>>> a client has not provided adequate Write list or Reply chunk
>>> resources to convey the reply.
>> 
>> I forgot that we made ERR_CHUNK ambiguous, apparently because of reluctance to add things to the XDR.  There is no pressing need to do this since the responder is aware of the difference and his management of the DRC can follow based on his knowledge.
>> 
>> However, in the context of Version Two it would be better if we avoided those ambiguities, given that we have lots of space for distinct error codes.
>> 
>>> In that case, it makes sense for the server to have added 
>>> the request to its DRC.
>> 
>> Agree that in the case in which the server has executed the request, it should add the request to the DRC.  In practical terms, there are not likely to be cases in which there is a non-idempotent request with a reply longer than 1K.
> 
> For NFSv3, that is largely true.
> 
> For NFSv4, I believe a large reply to a non-idempotent request is
> possible, and may even be common. Any time a client does something
> like this:
> 
>  { SEQUENCE, PUTFH, SETATTR, GETATTR }
> 
> Where the GETATTR requests an ACL or security label, is problematic
> if the client does not estimate the reply buffer size correctly.
> 
> However, this case is for when the client has a bug; it's not a
> case where we expect one or the other side to perform heroic
> recovery. ERR_CHUNK would terminate the RPC on the client, which
> would very likely return EIO to the application. I think that's
> about the sanest outcome we can expect.
> 
> 
>> As far as the session case, the server will consider the request executed but the client does not have a reply containing the slot and sequence.  To deal with that case, he would have to use rdma_xid in the message with the ERR_CHUNK to get the needed context and so conclude that the slot was available for reuse.
> 
> This feels like something that belongs in 5667bis. I'm no real
> expert on session behavior. Can you sketch some text that can be
> added?
> 
> 
>>> What I propose is that if the first READ_PLUS returns 
>>> NFS4_CONTENT_HOLE, the server would return an empty first
>>> Write chunk. Then the second READ_PLUS result always 
>>> lines up with the second Write chunk, which IMO is much better > for clients.
>> 
>> I'm OK with this but I think you will need to adjust the text to reflect the fact that READ_PLUS can return an array of read_plus_content's, although, in practice, those that return more than one are extremely rare.
> 
> I hadn't realized READ_PLUS returned an array.
> 
> If an NFS server is allowed to structure its reply in a way that
> the client cannot predict, then I think we'll have to limit the
> way READ_PLUS uses DDP. I propose these rules:
> 
> - The client can provide no more than one Write chunk if it expects
> NFS4_CONTENT_DATA. (No Write chunk or an empty Write chunk, following
> the previous rules, would be for when the client predicts that the
> reply can go inline).
> 
> - If that Write chunk is non-empty, it MUST be large enough to
> receive all expected payload bytes in a single NFS4_CONTENT_DATA
> element.
> 
> - The server uses that Write chunk for the first array element that
> has an NFS4_CONTENT_DATA arm.
> 
> 
> Then we have a choice, depending on whether it is more desirable
> to return data in a single round-trip, or more desirable to preserve
> holes. Either:
> 
> - If the server finds that the array has grown larger than can be
> returned inline or via the supplied Reply chunk, it MUST return
> the payload data in a single NFS4_CONTENT_DATA element via the
> provided Write chunk.
> 
> Or:
> 
> - The server MUST return as much payload as it can fit within the
> resources provided by the client, and return it as a short READ
> result.

Sorry, that's not clear. I mean a "short read result": In other
words, the server indicates that it didn't return all the bytes
the client requested.


> The client is responsible for retrying the READ_PLUS to
> read the remaining payload.
> 
> Somehow we have to deal with the case where the server cannot fit
> any of the payload in the client-provided resources.
> 
> 
> READ_PLUS is actually a poor fit for offloaded DDP anyway. The whole
> point of offload is that the client has to do no work; the payload
> arrives in its memory without any effort on its part.
> 
> I would just as soon require that, on RDMA transports, READ_PLUS
> returns only a hole or exactly one contiguous piece of content.
> 
> 
> --
> Chuck Lever
> 
> 
> 
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4

--
Chuck Lever