Re: [nfsv4] request for xattr as a WG item

"McDonald, Alex" <Alex.Mcdonald@netapp.com> Tue, 24 March 2015 22:09 UTC

Return-Path: <Alex.Mcdonald@netapp.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2AF0A1A1B28 for <nfsv4@ietfa.amsl.com>; Tue, 24 Mar 2015 15:09:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.911
X-Spam-Level:
X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ypgWcVOGbzpT for <nfsv4@ietfa.amsl.com>; Tue, 24 Mar 2015 15:09:16 -0700 (PDT)
Received: from mx143.netapp.com (mx143.netapp.com [216.240.21.24]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1D6C91A1A66 for <nfsv4@ietf.org>; Tue, 24 Mar 2015 15:09:16 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="5.11,460,1422950400"; d="scan'208";a="31449150"
Received: from hioexcmbx04-prd.hq.netapp.com ([10.122.105.37]) by mx143-out.netapp.com with ESMTP; 24 Mar 2015 15:04:14 -0700
Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.995.29; Tue, 24 Mar 2015 15:04:14 -0700
Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::f07f:691d:89d:53b7%21]) with mapi id 15.00.0995.031; Tue, 24 Mar 2015 15:04:14 -0700
From: "McDonald, Alex" <Alex.Mcdonald@netapp.com>
To: Marc Eshel <eshel@us.ibm.com>
Thread-Topic: [nfsv4] request for xattr as a WG item
Thread-Index: AQHP4ZykVQpkTj5F2kK8KWzXveykcpwj7x2AgAAbLACAAAM1gIAACZGAgAHDvYCAAAb+gIABTPcAgAARzgCAACitgIDzT+wAgBKQqQD//5mYUIAAgLaA///IN1A=
Date: Tue, 24 Mar 2015 22:04:13 +0000
Message-ID: <a8c0706b3d77485ca05b9c0c1ada7849@hioexcmbx07-prd.hq.netapp.com>
References: <CADaq8je5x2UjDcQ=g8y188Qnjpv=Ge7EuaihOLbtRnK7P+x_kg@mail.gmail.com> <20141003190419.GC4701@fieldses.org>, <OF66DFA0A8.35491496-ON88257D66.006A8A33-88257D66.006DDA48@us.ibm.com> <7308296a2dad41339eb25141ec7de137@BL2PR03MB369.namprd03.prod.outlook.com> <OF689B84FC.D46BD76F-ON88257D69.0067EFFF- <64A81EEC-E8B2-423D-9EE5-C0CD8A2BCC96@primarydata.com> <OFA858D342.4C470B0D-ON88257D69.00714B10-88257D69.007 <B36E413B-2E4D-46DE-ACAD-1E5EF0CE315B@primarydata.com> <54348E7F.6030305@davequigley.com> <OFDF03B46F.85E8C096-ON88257D6B.000 <abf139a289cb41a785e42f131a8a5034@BL2PR03MB369.namprd03.prod.outlook.com> <OF1AA9EA23.7239BC03-ON88257D6B.0077C1A4-88257D6B.0 <e0d3d84ac21146e0857e6a84e4fed151@BL2PR03MB369.namprd03.prod.outlook.com> <e24c4fde0d5346e4a3c0f8053baa3ce8@hioexcmbx07-prd.hq.netapp.com> <OFDB1FE0C2.EF4CDBC4-ON88257E12.00602183-88257E12.0060B21C@us.ibm.com>
In-Reply-To: <OFDB1FE0C2.EF4CDBC4-ON88257E12.00602183-88257E12.0060B21C@us.ibm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.120.60.37]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Archived-At: <http://mailarchive.ietf.org/arch/msg/nfsv4/8DFcghOB41wEpyJDTegDsEPXcq8>
Cc: "nfsv4@ietf.org" <nfsv4@ietf.org>
Subject: Re: [nfsv4] request for xattr as a WG item
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Mar 2015 22:09:20 -0000

Thanks Marc, that was what I suspected; I hadn't read the draft proposal. My apologies.

Specifically addressing https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01

5.1.1 and 5.1.2 assume that the underlying file system is uniform; either a single file system or a union of file systems that have maxxattrsize and xattrsize values that match.  

5.2.2 Caching suggests that xattr changes will be infrequent and that changes can be propagated synchronously. (In fact, the text kind of hopes that they're done synchronously, and I don’t see what justifies that hope.) How will this work in geographically disperse file systems that may have significant delays?

5.2.4.3 DESCRIPTION (for SETXATTR) describes atomicity considerations and imposes at least an in-order requirement for xattr deletions, updates or creations. A successful SETXATTR should (but is not required to) set time_modify. Although each xattr operation (update or delete) can be reported as a success or failure individually, is time_modify updated on partial failure? How does that reconcile with the use of time_access to detect cache incoherency?

5.2.4.3 DESCRIPTION (for SETXATTR) How should the server handle the case where an attribute is deleted by client A and then SETXATTRed  with SETXATTR4_REPLACE by client B?  

I'll now turn to my questions.

a.       Security. Do the metadata/xattrs have the same security as the base data?

The xattrs proposal says no; but protected by ACE (5.3).

b.      Searchability. Can users expect to search for keys or keys and values, and how would the protocol support it?

Not without difficulty or at least, it's not clear to me how to support searching for user.colour="red" without client support, i.e. requesting attributes that don't match and discarding them. I can't see a way to inform the server of this type of request. (The whole area of server side searching for metadata is complex and problematic, since a lot of the use cases for it require matching that isn’t for equality.)

c.       Placement. Does the proposal imply colocation of metadata/xattrs with the base data, or are there requirements where they might  be divided?

Not specified; server defines.

d.      Updates and locking. Is the metadata/xattrs supported by locking or some “eventual consistency” model?

There are descriptions about atomicity and cache coherence, but it's not clear to me how it is to be achieved.

e.      Versioning. Is it anticipated to provide support for this?

Not discussed, but it appears not (or rather, there's nothing stopping the server doing this, but no way in the protocol to retrieve it). Important for audit.

f.        Namespace. Do metadata/xattrs require internal and/or external namespace(s) and extension to filenames? Will metatdat/xattrs be addressable via URIs?

No. Client addresses via file handle & namespace.xattribute key.

g.       Streams. Is it possible to treat the metadata/xattrs as a file in its own right?
No. Named attributes only.

I'd appreciate an update on (b)  (d) and (e).  

The proposed use cases seem limited in their scope and potential use. For instance, a broadcaster of my acquaintance wishes to store multiple closed caption streams in various languages along with the video. This captioning also provides them with searchable metadata as it is text that describes (almost to the second) the content of the video. That would require larger value fields than xattrs appears to be capable of supporting. 

There's an argument that you make in the spec that these types of attribute are named attributes, which are stream like and hence not xattrs. Xattrs appear to only have two differences, one of which is a downside; size, which is arbitrarily and server implementation limited, and atomicity. And atomicity I have problems with (see the questions above).

Thanks for your patience & time in replying to this. 

Alex McDonald
Standards & Industry Associations Group
CTO Office
NetApp
+44 7795 046686 Mobile Phone
alexmc@netapp.com
twitter: @alextangent
Follow us: 


> -----Original Message-----
> From: Marc Eshel [mailto:eshel@us.ibm.com]
> Sent: 24 March 2015 17:36
> To: McDonald, Alex
> Cc: Marc Eshel; nfsv4@ietf.org; Spencer Shepler
> Subject: RE: [nfsv4] request for xattr as a WG item
> 
> Hi Alex,
> 
> Please read the following draft and than see what questions you have left.
> 
> https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01
> 
> Thanks, Marc.
> 
> 
> 
> From:   "McDonald, Alex" <Alex.Mcdonald@netapp.com>
> To:     Spencer Shepler <sshepler@microsoft.com>, Marc
> Eshel/Almaden/IBM@IBMUS, "nfsv4@ietf.org" <nfsv4@ietf.org>
> Date:   03/24/2015 10:16 AM
> Subject:        RE: [nfsv4] request for xattr as a WG item
> 
> 
> 
> This is feedback based not on implementation possibilities or issues but on
> the role of extended attributes and metadata.
> 
> There is a great deal of talk about extended attributes, when what appears
> to be meant is data that is
> 
> a.       retrievable by key
> b.      bound to a root (the base data or value)
> 
> Given that the proposal would appear to require keys and values that are not
> fixed or limited in size, I have several questions that aren’t answered here
> (and I apologise if there is some other part of the proposal that I’ve not seen
> or read that explains this).
> 
> a.       Security. Do the metadata/xattrs have the same security as the
> base data?
> b.      Searchability. Can users expect to search for keys or keys and
> values, and how would the protocol support it?
> c.       Placement. Does the proposal imply colocation of metadata/xattrs
> with the base data, or are there requirements where they might  be divided?
> d.      Updates and locking. Is the metadata/xattrs supported by locking
> or some “eventual consistency” model?
> e.      Versioning. Is it anticipated to provide support for this?
> f.        Namespace. Do metadata/xattrs require internal and/or external
> namespace(s) and extension to filenames? Will metatdat/xattrs be
> addressable via URIs?
> g.       Streams. Is it possible to treat the metadata/xattrs as a file in
> its own right?
> 
> Thanks.
> 
> Alex McDonald
> Standards & Industry Associations Group
> CTO Office
> NetApp
> +44 7795 046686 Mobile Phone
> alexmc@netapp.com
> twitter: @alextangent
> Follow us:
> 
> 
> From: nfsv4 [mailto:nfsv4-bounces@ietf.org] On Behalf Of Spencer Shepler
> Sent: 24 March 2015 16:02
> To: Marc Eshel; nfsv4@ietf.org
> Subject: Re: [nfsv4] request for xattr as a WG item
> 
> 
> Any feedback for Marc?  I would like to close this week on the decision to
> move this to WG work….
> 
> From: Marc Eshel [mailto:eshel@us.ibm.com]
> Sent: Thursday, March 12, 2015 1:31 PM
> To: Spencer Shepler
> Cc: Marc Eshel; nfsv4@ietf.org
> Subject: RE: [nfsv4] request for xattr as a WG item
> 
> This a sample usage of xattr that was requested to help reaching consensus
> for the need of xattr and move the draft to a WG I-D Thanks, Marc.
> 
> 
> Background:
> While extended attributes ( a.k.a. xaatr or EA) are around in most filesystems
> for quite some time, standard applications that uses them are not that
> common.
> It seems that one of the main inhibitors for that is lack of support in standard
> file sharing protocols ( NFS for example).
> This document goal is to describe use cases in which customers are using
> xattr ( over proprietary protocols or CIFS/SMB) and would probably do so
> over NFS if this feature would be available.
> 
> Extended attributed in Digital media:
> One of the main use cases  that comes to mind when discussing xattr as a
> mean to store extended information as part of a file metadata is in the digital
> media industry.
> One of the biggest challenges in this industry is managing content, which
> goes through various platforms and phases during the processing workflow.
> If in the past, media content was mostly produced by legacy film based
> sources, mostly of professional grade – now days, media content may be
> produced by different authors ( professional and amateurs) using different
> tools ( cell phones, security cams, professional grade equipment etc.)
> following different standards and platforms ( Windows based, Linux,
> Android, Osx, IOS etc.), needs to be processed using various tools ( video
> editing, image processing etc.) and shared with the via various  platforms (
> web, TV etc.) to various end devices ( different resolution, quality
> etc.)
> Thus, keeping content metadata in a standard form through the lifetime of
> the content is highly required – but in reality, lack of standards is making this
> type of content management a nightmare, especially using open tools (1)
> 
> Example use case – digital video:
>  Let's looks on this example workflow:
>  1.     Video is being captured on a fixed camera into its embedded Linux
> box
>  2.     Video is edited using Video editing software on Mac OSX box
>  3.     Video is reviewed by the reporter  on his Windows desktop
>  4.     Video is transmitted on air – to consumers TVs using proprietary
> system
>  5.     Video is shared on the TV station web site
> 
> So, lets look again in the above example and try to add some more metadata
> context:
> ---------------------------------------------------------------------------
> 
> | Workflow stage                | attribute/tags              | comments
>                   |
> ---------------------------------------------------------------------------
> 
> | Capture                        |Location, time              |   |
> ---------------------------------------------------------------------------
> 
> | Editing                        | Editor name, change tracking,        |
>                              |
> |                                        | content comments, generation
> |                                |
> ---------------------------------------------------------------------------
> 
> | Review                                | Speaker notes, content tags    |
>                                |
> ---------------------------------------------------------------------------
> 
> | On air transmission| Quality/resolution tags(HD/SD)        | Multiple
> version        |
> ---------------------------------------------------------------------------
> 
> | Web sharing                | Copyright/trademark,additional        |
> Multiple versions        |
> |                                        | quality/resolution tags  |
>            |
> ---------------------------------------------------------------------------
> 
> 
> 
> As can be seen above, not only that the content itself is being
> transformed and enhanced during its lifetime – its metadata is enriched
> and needs to be accessed by multiple applications on multiple platforms.
> Today, for the most part, a 3rd party application is required in order to
> manage the content metadata and/or proprietary filesystems that can store
> those attributed with the file
>  – potentially on all the platforms. And even with those, there might be
> compatibility issues (2)
> 
> Current solutions:
> So, it seems what some customers are doing is using SMB/CIFS for the
> purpose of sharing the data between platforms. The drawback of that is
> that SMB/CIFS is somewhat not working well.
> Attempts to use NFS ( as it looks like the best logical solution) – don't
> work ( and even breaks the apps completely) as it don't support xattr (3)
> 
> Conclusion
> In today's world, managing the large variety of content becomes a
> challenging task. Extended attributes can ( and do ) assist with that –
> but unfortunately they cannot be used with NFS due to lack of support.
> 
> Other examples:
> There are other cases in which xattr starts to gain traction, including:
>  1. HDFS:
> http://mapredit.blogspot.co.il/2014/07/xattr-are-coming-to-hdfs.html
>  2. Openmeta: https://code.google.com/p/openmeta/
> 
> 
> References:
> (1)
> http://www.strategyand.pwc.com/global/home/what-we-think/reports-
> white-papers/article-display/media-workflow-digital-era
> 
> (2) https://discussions.apple.com/message/19411219#19411219
> (3)http://superuser.com/questions/427581/osx-extended-attributes-and-
> nfs
> 
> 
> 
> 
> 
> From:        Spencer Shepler <sshepler@microsoft.com>
> To:        Marc Eshel/Almaden/IBM@IBMUS
> Cc:        "nfsv4@ietf.org" <nfsv4@ietf.org>
> Date:        10/08/2014 05:54 PM
> Subject:        RE: [nfsv4] request for xattr as a WG item
> 
> 
> 
> 
> 
> Marc,
> 
> I have not observed consensus as of yet -- I see positive comments and
> engagement and with use-case clarity we should be able to move forward.
> 
> The use-case item does not need to land in an I-D if you are looking for
> minimal overhead.  A email to the WG with a draft of what would land in
> the I-D would be sufficient to get feedback and to demonstrate consensus.
> 
> BTW: just to state the obvious, when an item becomes a WG I-D does not
> guarantee forward movement...
> 
> Spencer
> 
> > -----Original Message-----
> > From: Marc Eshel [mailto:eshel@us.ibm.com]
> > Sent: Wednesday, October 8, 2014 3:29 PM
> > To: Spencer Shepler
> > Cc: Dave Quigley; Marc Eshel; nfsv4@ietf.org; nfsv4
> > Subject: RE: [nfsv4] request for xattr as a WG item
> >
> > Hi Spencer,
> >
> > We have enough proposed changes from the feedback in the last few days
> that
> > a new update to the draft is needed, but after working on it for more
> than a
> > year and watching several rounds of similar discussions at different
> times on
> > the list and at IETF, I would like to hear some assurance from the
> chairs that
> > the WG will take it up as an item. We take it as an action item to
> update the
> > draft, but this is not a personal mission, and needs support from the
> > community. I just want confirmation that this is not a waste of our
> time.
> >
> > Thanks, Marc.
> >
> >
> >
> > From:   Spencer Shepler <sshepler@microsoft.com>
> > To:     Marc Eshel/Almaden/IBM@IBMUS, Dave Quigley
> > <dpquigl@davequigley.com>,
> > Cc:     "nfsv4@ietf.org" <nfsv4@ietf.org>, nfsv4 <nfsv4-bounces@ietf.org
> >
> > Date:   10/08/2014 02:25 PM
> > Subject:        RE: [nfsv4] request for xattr as a WG item
> >
> >
> >
> >
> > Marc,
> >
> > Do you have an idea of when an update to the I-D may be available?
> >
> > Spencer
> >
> > > -----Original Message-----
> > > From: nfsv4 [mailto:nfsv4-bounces@ietf.org] On Behalf Of Marc Eshel
> > > Sent: Tuesday, October 7, 2014 6:33 PM
> > > To: Dave Quigley
> > > Cc: nfsv4@ietf.org; nfsv4
> > > Subject: Re: [nfsv4] request for xattr as a WG item
> > >
> > > "nfsv4" <nfsv4-bounces@ietf.org> wrote on 10/07/2014 06:08:15 PM:
> > >
> > > > From: Dave Quigley <dpquigl@davequigley.com>
> > > > To: nfsv4@ietf.org,
> > > > Date: 10/07/2014 06:08 PM
> > > > Subject: Re: [nfsv4] request for xattr as a WG item Sent by: "nfsv4"
> > > > <nfsv4-bounces@ietf.org>
> > > >
> > > > On 10/6/2014 6:11 PM, Marc Eshel wrote:
> > > > >
> > > > > We added a section to the draft based on your request to explain
> > > > > why
> > > it is
> > > > > needed, but you are still not convinced. I think that you are
> > > > > looking
> > > for
> > > > > a killer application that will prove the need for xattr and I
> > > > > don't
> > > know
> > > > > of one. all I know is what we added to the draft and what
> > > > > customers
> > > tell
> > > > > us, but it is not enough for you. So I don't know how to convince
> > > > > you, others on the WG seem to agree that it is needed, I assume
> > > > > based on
> > > their
> > > > > knowledge and experience. I can not give you a name of an
> > > > > application
> > > that
> > > > > a customer is running because I don't know the name and even if I
> > > > > did
> > > it
> > > > > would mean nothing, all I can say that for example one application
> > > > > is
> > > used
> > > > > to tag files using xattr for future migration, it is like a policy
> > > > > for file movement among the different storage levels. Maybe you
> > > > > should
> > > talk to
> > > > > your customers and see if they needed it.
> > > > >
> > > >
> > > >
> > > > The thing is that just having a section justifying the feature is
> > > > not enough. I learned that when I started Labeled NFS. I started
> > > > with a requirements document. I then was asked to a use case
> > > > document, an impact study, a requirements document, and finally the
> > > > actual specification which got merged into 4.2. Compared to the
> > > > varied semantics of xattr implementations on different systems
> > > > security labels were easy. We punted on syntax and semantics and
> > > > abstracted that out into a separate document. We punted on
> > > > establishing a security model and
> > >
> > > > said it was out of scope since its really a platform specific
> > > > implementation detail.
> > > >
> > > > If you want a way of laying out your case for xattrs you can try
> > > > following what I started for Labeled NFS and what Tom continued and
> > > > finally shepherded through the 4.2 adoption process. We have
> > > > specific use cases in there for security labels. I'm sure you can
> > > > find real concrete examples for the equivalent of the user namespace
> > > > style xattrs from Linux.
> > >
> > > I agree that we have to add more to the section on justifying the need
> > for xattr,
> > > and we will, and invite again members of the WG to contribute to this
> > section,
> > > like Pranoop and others did. I don't think that the fact that you and
> > Tom had a
> > > hard time with Labeled NFS should imply that every new feature now
> > > must have a separate document to justify it. Where is this requirement
> > > coming from? I believe that we need to reach consensus in the WG and
> > > that
> > should be
> > > enough to add a new feature. Am I wrong with this assumption?
> > >
> > > >
> > > > Dave
> > > >
> > > > _______________________________________________
> > > > nfsv4 mailing list
> > > > nfsv4@ietf.org
> > > > https://www.ietf.org/mailman/listinfo/nfsv4
> > > >
> > >
> > > _______________________________________________
> > > nfsv4 mailing list
> > > nfsv4@ietf.org
> > > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> >