Return-Path: X-Original-To: gen-art@ietfa.amsl.com Delivered-To: gen-art@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8FDA21A1BFA for ; Tue, 2 Dec 2014 07:54:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EaWeTHBovb3V for ; Tue, 2 Dec 2014 07:54:33 -0800 (PST) Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 659881A1BCD for ; Tue, 2 Dec 2014 07:54:33 -0800 (PST) Received: by mail-pa0-f49.google.com with SMTP id eu11so13607419pac.22 for ; Tue, 02 Dec 2014 07:54:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=CkZRWS1Uo+skzkNKdDIIffrWbVdODFbZwqtz6uwPdtM=; b=brCDqVeYRpWiWEfih6iw2JgDNwk5iplrOQm25iUJGxssXmxpF1iC4y1Jqcwv4u48Y6 Fs/kCltc3lnkGfnZKxwVvcsxl5meMuYTisOfy02clcyl3ry1tSj2NMPXxBxaxKoSR0kC gqkhE3xnb9jWHwXET9wZJ+C5BvNKpcgGQQFQ7Jt1PEvOnx390WwHxNcN3ghoHx31+AXH fU2ySTixAzg+Hd1Po9mRy5xm8p9FKNvJ3ys7mADv6MUOz8zyQYE2L4xjRjScl5TcGsAw l0J1TAUDT6R10PMxw/MpTLG4e891jedvUisS+ccv+xo4BuOn6PRGDxycoLU9rRkhfWlX LPPg== X-Gm-Message-State: ALoCoQletP0+FkP7igXC8xa6s6421+D3deyZNN9a0GlLhxK9hf72STEmqmguUR1QMKWlGtYTGG9z X-Received: by 10.70.134.10 with SMTP id pg10mr111457890pdb.47.1417535672703; Tue, 02 Dec 2014 07:54:32 -0800 (PST) Received: from loghyr.internal.excfb.com (c-50-131-226-197.hsd1.ca.comcast.net. [50.131.226.197]) by mx.google.com with ESMTPSA id nt6sm20740730pdb.26.2014.12.02.07.54.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 02 Dec 2014 07:54:31 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) From: Thomas Haynes In-Reply-To: <547DC609.9070104@dial.pipex.com> Date: Tue, 2 Dec 2014 07:54:30 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <0DC6FCB2-903C-4AC0-92DC-2824519EF213@primarydata.com> References: <54627625.9050909@dial.pipex.com> <7E1914F4-413F-4F7E-BE9A-C941FBDD422B@primarydata.com> <5465143D.6020604@dial.pipex.com> <47860B2D-1977-4062-9775-10463F7030B9@primarydata.com>

<546B0F4F.3060504@dial.pipex.com> <547C7E2B.10305@dial.pipex.com> <547CC75C.7010909@dial.pipex.com> <1F2B554C-4C7C-4BF0-B078-AA7988F7B0AF@primarydata.com> <547DC609.9070104@dial.pipex.com> To: Elwyn Davies X-Mailer: Apple Mail (2.1993) Archived-At: http://mailarchive.ietf.org/arch/msg/gen-art/ws5FTvTYVcmQoVpDB0JhhI2E_Fc Cc: "gen-art >> General area reviewing team" , Barry Leiba , David Noveck Subject: Re: [Gen-art] Gen-art LC review of draft-ietf-nfsv4-rfc3530bis-33 X-BeenThere: gen-art@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "GEN-ART: General Area Review Team" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 15:54:36 -0000 > On Dec 2, 2014, at 6:00 AM, Elwyn Davies = wrote: >=20 > Hi. >=20 > Sorry to keep harping on about READDIR, but... >=20 > Tom's comment about opendir/closedir at the end of the mail nbelow = obviously triggered something in my brain overnight, 'cos I woke up = realizing that I wasn't correctly understanding the semantics here. >=20 > Please bear with while I parade my naivete :-( >=20 > Having refreshed my memory on the semantics of reading directories in = Unix and checked back on NFSv4, I realized that NFSv4 doesn't have the = equivalents of opendir and closedir. >=20 I was actually talking about on the client, aka opendir(3) and = closedir(3) on Linux. So while the protocol may have no notion of these, = these do exist and interoperate with the protocol. In Linux, we have: const struct file_operations nfs_dir_operations =3D { .llseek =3D nfs_llseek_dir, .read =3D generic_read_dir, .iterate =3D nfs_readdir, .open =3D nfs_opendir, .release =3D nfs_closedir, .fsync =3D nfs_fsync_dir, }; I.e., the client is aware of when the closedir(3) occurs. > Hence, NFS has no equivalent of the "directory stream" referred to in = the Unix manual pages. Furthermore you don't 'open' a directory in = NFSv4, you just give it the filehandle. >=20 > So, the case of the directory getting deleted under your feet between = READDIR calls presumably just results in a NFS4ERR_STALE error on the = next READDIR call and the issue of whether the cookie is valid is moot. = It is then up to the client to map this into local OS behaviour. >=20 > This means that only the directory modification case is relevant to = the cookie validity issue, and I understand the comment that David was = making yesterday more clearly now. >=20 > However the lack of a closedir equivalent means that the server has no = idea when the client has finished with the READDIR cookie(s). I think = it would be helpful to give some advice on what the server should do to = avoid a build up of probably useless state. >=20 > Given the usual usage of READDIR, I would expect that once a call has = returned eof<-TRUE, the client will in almost all cases be done and the = state for the (client,directory) pair can be discarded. However, there = might be cases where the call has to be rerun due to comms failure I = suppose, so maybe after a number of calls at eof? Presumably there also = ought to be some sort of timeout (like if the leases expire??) I have = no idea what current implementations do here. This is like any other call to the server which hangs - most likely an = EDELAY is passed back up to the caller on the client. >=20 > Crafting a couple of sentences to mention these three items (deletion, = modification and stale state) would IMO be helpful. It would certainly = have helped me! >=20 > Regards, > Elwyn >=20 > On 02/12/14 00:56, Tom Haynes wrote: >>=20 >>> On Dec 1, 2014, at 11:54 AM, Elwyn Davies = wrote: >>>=20 >>> Hi, David. >>>=20 >>> Thanks for the response. >>>=20 >>> A couple of comments below... >>>=20 >>> Regards, >>> Elwyn >>>=20 >>> On 01/12/14 18:50, David Noveck wrote: >>>> My view (remains) that if the cookie is not tied to the change >>>> attribute of the directory, this makes life difficult for both = the >>>> server (what does it send if the data has been changed and how = does >>>> it manage maintaining checkpoints -- where a readdir finished = --, in >>>> particular if the checkpoint was on an entry that has been = deleted), >>>> and the client. >>>>=20 >>>>=20 >>>> The cookie is not tied to the change attribute of the directory. >>>>=20 >>>> The purpose of the cookie is ensuring that you do a READDIR that >>>> encompasses multiple requests: >>>>=20 >>>> * entries that exist during the entire period of the = multi-requuest >>>> directory read appear exactly once in the result. >>>> * entries that exists during some part of the period of the >>>> multi-request directory read appear at most once in the result. >>>>=20 >>>>=20 >>>> Basically implementations scan through the directory as it exists = on >>>> disk and if there is a created entry, the client may or may not see = it, >>>> depending on where the the new entry happens to be put, with = respect to >>>> the ongoing scan. As far as deleted entries, the implementation = only >>>> has to be able to move on to the next entry if it encouneters an = entry >>>> marked as deleted. >>>=20 >>> OK, so the server has to be clever rather than telling the client to = restart the readdir -- I'm not sure this is the right trade-off since = the client has to cope with being told the cached cookie is useless (as = per Trond's input below) but I guess implementations work this way = already and it isn't my call. In that case I suggest that the last = sentence of s15.26.4, para 9, is modified to tell server implementers a = bit more clearly what they have to do. Suggestion: >>> OLD: >>> Ideally, the cookie value should not change if the directory is >>> modified since the client may be caching these values. >>> NEW: >>> The server SHOULD try to accept cookie values issued with READDIR >>> responses even if the directory has been modified between the >>> READDIR calls but MAY return NFS4ERR_NOT_VALID if this is not >>> possible as might be the case if the server has rebooted in the >>> interim. >>=20 >> I=E2=80=99ve taken this change. >>=20 >>>=20 >>> One nasty thought occurs to me regarding the idea that a client = caches >>> the cookie value(s): Is the server supposed to maintain the whole >>> sequence of cookies for a sequence of READDIRs on a single directory = so >>> the client can reissue some of the READDIR calls or is it fair = enough for the server to throw them away after the client has used them = once? And how long should the server hang onto cookie state in normal = operation? >>>=20 >>=20 >> No, it does not have to save them. >>=20 >> But the cookies are pretty persistent in the first place. I.e., use = some property of either the inode or the dirent to generate the cookie. >>=20 >> As such, the cookie state does not need to be held onto. >>=20 >> And for the life, it actually needs to be =E2=80=9Cstable=E2=80=9D = between the opendir() and closedir() call on the client. While we are = safe on a client reboot, a client could end up holding onto the DIR = pointer for a large indeterminate time. >>=20