Re: [nfsv4] persistent sessions and compound atomicity

[Rick Macklem <[e-mail]>]

On 12/9/2020 9:49 AM, J. Bruce Fields wrote:
> On Wed, Dec 09, 2020 at 08:55:03AM -0500, Tom Talpey wrote:
>> On 12/8/2020 7:26 PM, J. Bruce Fields wrote:
>>> I hadn't given much thought to this language in RFC 5661, and now that I
>>> do, it surprises me.  If you have any pointers to discussion I forgot or
>>> overlooked, I'd be interested:
>>>
>>>     https://tools.ietf.org/html/rfc5661#section-2.10.6.5
>>>
>>>     The execution of the sequence of operations (starting with
>>>     SEQUENCE) and placement of its results in the persistent cache
>>>     MUST be atomic.  If a client retries a sequence of operations
>>>     that was previously executed on the server, the only acceptable
>>>     outcomes are either the original cached reply or an indication
>>>     that the client ID or session has been lost (indicating a
>>>     catastrophic loss of the reply cache or a session that has been
>>>     deleted because the client failed to use the session for an
>>>     extended period of time).
>>>
I think this is what others are already saying, but just in case...

I would say that the ATOMIC refers to the individual operations of the
compound. In other words, it must cache the reply to each operation
when it is completed, in the persistent cache.

Then, if a retry of the compound is received after a server reboot, the
reply for the first K operations that were completed prior to the reboot
are generated from the cache and the remaining N-K are performed on
the file system to complete the reply. (At least for the non-idempotent
operations in the compound.)

For this to all work correctly, I think the server file system would need to perform
each non-idempotent operation ATOMICALLY as well, such that after the
reboot the file system is in either the pre-operation or post-operation state.

rick, who has no intention of ever trying to implement a persistent session cache.

>>>     A server could fail and restart in the middle of a COMPOUND
>>>     procedure that contains one or more non-idempotent or
>>>     idempotent-but-modifying operations. This creates an even higher
>>>     challenge for atomic execution and placement of results in the
>>>     reply cache....
>>>
>>> And I don't see why that MUST is necessary.  Why can't the server, on
>>> encountering a nonidempotent operation, store partial compound results
>>> in its reply cache together with whatever compound state (current
>>> filehandles, etc.) it needs to resume processing on restart?
>>
>> I think the atomicity is referring to the building of the response,
>> not the execution of the COMPOUND itself (which would be impossible,
>> generally).
>>
>> The main issue it's protecting is the server's response never changing.
>> The operation must always have one result. If the client were to replay
>> a request while the server is still executing the compound, and get a
>> partial answer, followed by the final answer, that would be very bad
>> indeed.
>
> Got it, thanks.
>
>>> Or, why not allow it to return the result so far and then DEADSESSION on
>>> any following operation?
>>
>> "So far"? That only makes sense if the compound were totally complete.
>> At which point, there is one and only one result.
>
> Hm, I'm not sure what you mean.  The server comes up from a crash and
> determines that it was in the middle of processing a compound, and that
> the last thing it does was succesfully execute a nonidempotent op in the
> middle of the compound.  it has to decide what it's going to use as the
> final reply for that compound.

In the persistent restarted server case, I agree with you. I thought
you were somehow arguing that the server could provide a reply from
cache, for an in-progress operation.

In the persistence case, I guess this means the server will build the
compound response in-progress, and keep it "secret" as long as it's
partial. On restart, it will need to sweep the cache, add any
necessary error for the compounds that didn't get executed, and mark
it complete and ready for responding whne the client replays.

That seems like a lot of fiddly stuff, but I guess it's important
for any non-idempotent operations. I bet there's more language that
needs tightening up.

Tom.


>
> --b.
>
>>
>> Tom.
>>
>>> One problem is that a create operation followed by a GETATTR really
>>> shouldn't fail in between the two--that'd make it hard to implement
>>> O_CREAT, for example.
>>>
>>> But it seems like a server should be able to avoid that without making
>>> every sa_cachethis operation completely atomic.
>>>
>>> Am I missing some reason for that MUST?
>>>
>>> --b.
>>>
>>> _______________________________________________
>>> nfsv4 mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/nfsv4
>>>
>>
>> _______________________________________________
>> nfsv4 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nfsv4
>

_______________________________________________
nfsv4 mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/nfsv4