Re: [nfsv4] request for xattr as a WG item

[Marc Eshel <[e-mail]>]

"McDonald, Alex" <[e-mail]> wrote on 03/24/2015 03:04:13 
PM:

> From: "McDonald, Alex" <[e-mail]>
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "[e-mail]" <[e-mail]>, Spencer Shepler 
> <[e-mail]>
> Date: 03/24/2015 03:04 PM
> Subject: RE: [nfsv4] request for xattr as a WG item
> 
> Thanks Marc, that was what I suspected; I hadn't read the draft 
> proposal. My apologies.
> 
> Specifically addressing 
https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01
> 
> 5.1.1 and 5.1.2 assume that the underlying file system is uniform; 
> either a single file system or a union of file systems that have 
> maxxattrsize and xattrsize values that match. 

yes.

> 5.2.2 Caching suggests that xattr changes will be infrequent and 
> that changes can be propagated synchronously. (In fact, the text 
> kind of hopes that they're done synchronously, and I don’t see what 
> justifies that hope.) How will this work in geographically disperse 
> file systems that may have significant delays?

This what we observed for use cases, typically files are tagged with some 
xattr and don't change. This true for setting of other attributes over 
NFS. What are you suggesting for use of xattr and the synchronous setting 
assumption.
 
> 
> 5.2.4.3 DESCRIPTION (for SETXATTR) describes atomicity 
> considerations and imposes at least an in-order requirement for 
> xattr deletions, updates or creations. A successful SETXATTR should 
> (but is not required to) set time_modify. Although each xattr 
> operation (update or delete) can be reported as a success or failure
> individually, is time_modify updated on partial failure? How does 
> that reconcile with the use of time_access to detect cache incoherency?

We can make it more clear in the draft but any change even partial change 
to xattr SHOULD change the file time_modify and change attributes

> 
> 5.2.4.3 DESCRIPTION (for SETXATTR) How should the server handle the 
> case where an attribute is deleted by client A and then SETXATTRed 
> with SETXATTR4_REPLACE by client B? 

I believe it should fail if client B request don't find the attribute that 
it is trying to replace.

> 
> I'll now turn to my questions.
> 
> a.       Security. Do the metadata/xattrs have the same security as 
> the base data?
> 
> The xattrs proposal says no; but protected by ACE (5.3).

It should default to the same protection as other attributes unless you 
chose the additional ACE extention.

> 
> b.      Searchability. Can users expect to search for keys or keys 
> and values, and how would the protocol support it?
> 
> Not without difficulty or at least, it's not clear to me how to 
> support searching for user.colour="red" without client support, i.e.
> requesting attributes that don't match and discarding them. I can't 
> see a way to inform the server of this type of request. (The whole 
> area of server side searching for metadata is complex and 
> problematic, since a lot of the use cases for it require matching 
> that isn’t for equality.)

Searching for xattr is not part of this draft.

> 
> c.       Placement. Does the proposal imply colocation of metadata/
> xattrs with the base data, or are there requirements where they 
> might  be divided?

This is implementation decision of the server and the draft has no 
requirements here.

> 
> Not specified; server defines.
> 
> d.      Updates and locking. Is the metadata/xattrs supported by 
> locking or some “eventual consistency” model?
> 
> There are descriptions about atomicity and cache coherence, but it's
> not clear to me how it is to be achieved.
> 

While the SETXATTR request MAY contain multiple attribute keys and/or 
values to be changed for a file, this does not impose any atomicity 
considerations on the server implementation.


> e.      Versioning. Is it anticipated to provide support for this?
> 
> Not discussed, but it appears not (or rather, there's nothing 
> stopping the server doing this, but no way in the protocol to 
> retrieve it). Important for audit.

We can extent the protocol to allow for it.

> 
> f.        Namespace. Do metadata/xattrs require internal and/or 
> external namespace(s) and extension to filenames? Will metatdat/
> xattrs be addressable via URIs?
> 
> No. Client addresses via file handle & namespace.xattribute key.

The draft doesn't include namespaces. 

> 
> g.       Streams. Is it possible to treat the metadata/xattrs as a 
> file in its own right?
> No. Named attributes only.
> 
> I'd appreciate an update on (b)  (d) and (e). 
> 
> The proposed use cases seem limited in their scope and potential 
> use. For instance, a broadcaster of my acquaintance wishes to store 
> multiple closed caption streams in various languages along with the 
> video. This captioning also provides them with searchable metadata 
> as it is text that describes (almost to the second) the content of 
> the video. That would require larger value fields than xattrs 
> appears to be capable of supporting. 
> 
> There's an argument that you make in the spec that these types of 
> attribute are named attributes, which are stream like and hence not 
> xattrs. Xattrs appear to only have two differences, one of which is 
> a downside; size, which is arbitrarily and server implementation 
> limited, and atomicity. And atomicity I have problems with (see the 
> questions above).


As you know NFS protocol includes named attributes which are better suited 
for the application that you describe. Like said in the draft the 
objective here is add support for applications that requires more of a 
file tagging mechanism something that is available on most modern file 
systems.

We have other comments from other reviews, on how to improve this 
protocol, that we plan to incorporate into the a WG draft when there is 
agreement that we are ready for one.

Thanks, Marc. 
 
> 
> Thanks for your patience & time in replying to this. 
> 
> Alex McDonald
> Standards & Industry Associations Group
> CTO Office
> NetApp
> [phone] Mobile Phone
> [e-mail]
> twitter: @alextangent
> Follow us: 
> 
> 
> > -----Original Message-----
> > From: Marc Eshel [mailto:[e-mail]]
> > Sent: 24 March 2015 17:36
> > To: McDonald, Alex
> > Cc: Marc Eshel; [e-mail]; Spencer Shepler
> > Subject: RE: [nfsv4] request for xattr as a WG item
> > 
> > Hi Alex,
> > 
> > Please read the following draft and than see what questions you have 
left.
> > 
> > https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01
> > 
> > Thanks, Marc.
> > 
> > 
> > 
> > From:   "McDonald, Alex" <[e-mail]>
> > To:     Spencer Shepler <[e-mail]>, Marc
> > Eshel/Almaden/IBM@IBMUS, "[e-mail]" <[e-mail]>
> > Date:   03/24/2015 10:16 AM
> > Subject:        RE: [nfsv4] request for xattr as a WG item
> > 
> > 
> > 
> > This is feedback based not on implementation possibilities or issues 
but on
> > the role of extended attributes and metadata.
> > 
> > There is a great deal of talk about extended attributes, when what 
appears
> > to be meant is data that is
> > 
> > a.       retrievable by key
> > b.      bound to a root (the base data or value)
> > 
> > Given that the proposal would appear to require keys and values that 
are not
> > fixed or limited in size, I have several questions that aren’t 
answered here
> > (and I apologise if there is some other part of the proposal that 
> I’ve not seen
> > or read that explains this).
> > 
> > a.       Security. Do the metadata/xattrs have the same security as 
the
> > base data?
> > b.      Searchability. Can users expect to search for keys or keys and
> > values, and how would the protocol support it?
> > c.       Placement. Does the proposal imply colocation of 
metadata/xattrs
> > with the base data, or are there requirements where they might  be 
divided?
> > d.      Updates and locking. Is the metadata/xattrs supported by 
locking
> > or some “eventual consistency” model?
> > e.      Versioning. Is it anticipated to provide support for this?
> > f.        Namespace. Do metadata/xattrs require internal and/or 
external
> > namespace(s) and extension to filenames? Will metatdat/xattrs be
> > addressable via URIs?
> > g.       Streams. Is it possible to treat the metadata/xattrs as a 
file in
> > its own right?
> > 
> > Thanks.
> > 
> > Alex McDonald
> > Standards & Industry Associations Group
> > CTO Office
> > NetApp
> > [phone] Mobile Phone
> > [e-mail]
> > twitter: @alextangent
> > Follow us:
> > 
> > 
> > From: nfsv4 [mailto:[e-mail]] On Behalf Of Spencer 
Shepler
> > Sent: 24 March 2015 16:02
> > To: Marc Eshel; [e-mail]
> > Subject: Re: [nfsv4] request for xattr as a WG item
> > 
> > 
> > Any feedback for Marc?  I would like to close this week on the 
decision to
> > move this to WG work….
> > 
> > From: Marc Eshel [mailto:[e-mail]]
> > Sent: Thursday, March 12, 2015 1:31 PM
> > To: Spencer Shepler
> > Cc: Marc Eshel; [e-mail]
> > Subject: RE: [nfsv4] request for xattr as a WG item
> > 
> > This a sample usage of xattr that was requested to help reaching 
consensus
> > for the need of xattr and move the draft to a WG I-D Thanks, Marc.
> > 
> > 
> > Background:
> > While extended attributes ( a.k.a. xaatr or EA) are around in most
> filesystems
> > for quite some time, standard applications that uses them are not that
> > common.
> > It seems that one of the main inhibitors for that is lack of 
> support in standard
> > file sharing protocols ( NFS for example).
> > This document goal is to describe use cases in which customers are 
using
> > xattr ( over proprietary protocols or CIFS/SMB) and would probably do 
so
> > over NFS if this feature would be available.
> > 
> > Extended attributed in Digital media:
> > One of the main use cases  that comes to mind when discussing xattr as 
a
> > mean to store extended information as part of a file metadata is 
> in the digital
> > media industry.
> > One of the biggest challenges in this industry is managing content, 
which
> > goes through various platforms and phases during the processing 
workflow.
> > If in the past, media content was mostly produced by legacy film based
> > sources, mostly of professional grade – now days, media content may be
> > produced by different authors ( professional and amateurs) using 
different
> > tools ( cell phones, security cams, professional grade equipment etc.)
> > following different standards and platforms ( Windows based, Linux,
> > Android, Osx, IOS etc.), needs to be processed using various tools ( 
video
> > editing, image processing etc.) and shared with the via various 
platforms (
> > web, TV etc.) to various end devices ( different resolution, quality
> > etc.)
> > Thus, keeping content metadata in a standard form through the lifetime 
of
> > the content is highly required – but in reality, lack of standards
> is making this
> > type of content management a nightmare, especially using open tools 
(1)
> > 
> > Example use case – digital video:
> >  Let's looks on this example workflow:
> >  1.     Video is being captured on a fixed camera into its embedded 
Linux
> > box
> >  2.     Video is edited using Video editing software on Mac OSX box
> >  3.     Video is reviewed by the reporter  on his Windows desktop
> >  4.     Video is transmitted on air – to consumers TVs using 
proprietary
> > system
> >  5.     Video is shared on the TV station web site
> > 
> > So, lets look again in the above example and try to add some more 
metadata
> > context:
> > 
---------------------------------------------------------------------------
> > 
> > | Workflow stage                | attribute/tags              | 
comments
> >                   |
> > 
---------------------------------------------------------------------------
> > 
> > | Capture                        |Location, time              |   |
> > 
---------------------------------------------------------------------------
> > 
> > | Editing                        | Editor name, change tracking,  |
> >                              |
> > |                                        | content comments, 
generation
> > |                                |
> > 
---------------------------------------------------------------------------
> > 
> > | Review                                | Speaker notes, content tags  
 |
> >                                |
> > 
---------------------------------------------------------------------------
> > 
> > | On air transmission| Quality/resolution tags(HD/SD)        | 
Multiple
> > version        |
> > 
---------------------------------------------------------------------------
> > 
> > | Web sharing                | Copyright/trademark,additional        |
> > Multiple versions        |
> > |                                        | quality/resolution tags  |
> >            |
> > 
---------------------------------------------------------------------------
> > 
> > 
> > 
> > As can be seen above, not only that the content itself is being
> > transformed and enhanced during its lifetime – its metadata is 
enriched
> > and needs to be accessed by multiple applications on multiple 
platforms.
> > Today, for the most part, a 3rd party application is required in order 
to
> > manage the content metadata and/or proprietary filesystems that can 
store
> > those attributed with the file
> >  – potentially on all the platforms. And even with those, there might 
be
> > compatibility issues (2)
> > 
> > Current solutions:
> > So, it seems what some customers are doing is using SMB/CIFS for the
> > purpose of sharing the data between platforms. The drawback of that is
> > that SMB/CIFS is somewhat not working well.
> > Attempts to use NFS ( as it looks like the best logical solution) – 
don't
> > work ( and even breaks the apps completely) as it don't support xattr 
(3)
> > 
> > Conclusion
> > In today's world, managing the large variety of content becomes a
> > challenging task. Extended attributes can ( and do ) assist with that 
–
> > but unfortunately they cannot be used with NFS due to lack of support.
> > 
> > Other examples:
> > There are other cases in which xattr starts to gain traction, 
including:
> >  1. HDFS:
> > http://mapredit.blogspot.co.il/2014/07/xattr-are-coming-to-hdfs.html
> >  2. Openmeta: https://code.google.com/p/openmeta/
> > 
> > 
> > References:
> > (1)
> > http://www.strategyand.pwc.com/global/home/what-we-think/reports-
> > white-papers/article-display/media-workflow-digital-era
> > 
> > (2) https://discussions.apple.com/message/19411219#19411219
> > (3)http://superuser.com/questions/427581/osx-extended-attributes-and-
> > nfs
> > 
> > 
> > 
> > 
> > 
> > From:        Spencer Shepler <[e-mail]>
> > To:        Marc Eshel/Almaden/IBM@IBMUS
> > Cc:        "[e-mail]" <[e-mail]>
> > Date:        10/08/2014 05:54 PM
> > Subject:        RE: [nfsv4] request for xattr as a WG item
> > 
> > 
> > 
> > 
> > 
> > Marc,
> > 
> > I have not observed consensus as of yet -- I see positive comments and
> > engagement and with use-case clarity we should be able to move 
forward.
> > 
> > The use-case item does not need to land in an I-D if you are looking 
for
> > minimal overhead.  A email to the WG with a draft of what would land 
in
> > the I-D would be sufficient to get feedback and to demonstrate 
consensus.
> > 
> > BTW: just to state the obvious, when an item becomes a WG I-D does not
> > guarantee forward movement...
> > 
> > Spencer
> > 
> > > -----Original Message-----
> > > From: Marc Eshel [mailto:[e-mail]]
> > > Sent: Wednesday, October 8, 2014 3:29 PM
> > > To: Spencer Shepler
> > > Cc: Dave Quigley; Marc Eshel; [e-mail]; nfsv4
> > > Subject: RE: [nfsv4] request for xattr as a WG item
> > >
> > > Hi Spencer,
> > >
> > > We have enough proposed changes from the feedback in the last few 
days
> > that
> > > a new update to the draft is needed, but after working on it for 
more
> > than a
> > > year and watching several rounds of similar discussions at different
> > times on
> > > the list and at IETF, I would like to hear some assurance from the
> > chairs that
> > > the WG will take it up as an item. We take it as an action item to
> > update the
> > > draft, but this is not a personal mission, and needs support from 
the
> > > community. I just want confirmation that this is not a waste of our
> > time.
> > >
> > > Thanks, Marc.
> > >
> > >
> > >
> > > From:   Spencer Shepler <[e-mail]>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS, Dave Quigley
> > > <[e-mail]>,
> > > Cc:     "[e-mail]" <[e-mail]>, nfsv4 
<[e-mail]
> > >
> > > Date:   10/08/2014 02:25 PM
> > > Subject:        RE: [nfsv4] request for xattr as a WG item
> > >
> > >
> > >
> > >
> > > Marc,
> > >
> > > Do you have an idea of when an update to the I-D may be available?
> > >
> > > Spencer
> > >
> > > > -----Original Message-----
> > > > From: nfsv4 [mailto:[e-mail]] On Behalf Of Marc 
Eshel
> > > > Sent: Tuesday, October 7, 2014 6:33 PM
> > > > To: Dave Quigley
> > > > Cc: [e-mail]; nfsv4
> > > > Subject: Re: [nfsv4] request for xattr as a WG item
> > > >
> > > > "nfsv4" <[e-mail]> wrote on 10/07/2014 06:08:15 PM:
> > > >
> > > > > From: Dave Quigley <[e-mail]>
> > > > > To: [e-mail],
> > > > > Date: 10/07/2014 06:08 PM
> > > > > Subject: Re: [nfsv4] request for xattr as a WG item Sent by: 
"nfsv4"
> > > > > <[e-mail]>
> > > > >
> > > > > On 10/6/2014 6:11 PM, Marc Eshel wrote:
> > > > > >
> > > > > > We added a section to the draft based on your request to 
explain
> > > > > > why
> > > > it is
> > > > > > needed, but you are still not convinced. I think that you are
> > > > > > looking
> > > > for
> > > > > > a killer application that will prove the need for xattr and I
> > > > > > don't
> > > > know
> > > > > > of one. all I know is what we added to the draft and what
> > > > > > customers
> > > > tell
> > > > > > us, but it is not enough for you. So I don't know how to 
convince
> > > > > > you, others on the WG seem to agree that it is needed, I 
assume
> > > > > > based on
> > > > their
> > > > > > knowledge and experience. I can not give you a name of an
> > > > > > application
> > > > that
> > > > > > a customer is running because I don't know the name and even 
if I
> > > > > > did
> > > > it
> > > > > > would mean nothing, all I can say that for example one 
application
> > > > > > is
> > > > used
> > > > > > to tag files using xattr for future migration, it is like a 
policy
> > > > > > for file movement among the different storage levels. Maybe 
you
> > > > > > should
> > > > talk to
> > > > > > your customers and see if they needed it.
> > > > > >
> > > > >
> > > > >
> > > > > The thing is that just having a section justifying the feature 
is
> > > > > not enough. I learned that when I started Labeled NFS. I started
> > > > > with a requirements document. I then was asked to a use case
> > > > > document, an impact study, a requirements document, and finally 
the
> > > > > actual specification which got merged into 4.2. Compared to the
> > > > > varied semantics of xattr implementations on different systems
> > > > > security labels were easy. We punted on syntax and semantics and
> > > > > abstracted that out into a separate document. We punted on
> > > > > establishing a security model and
> > > >
> > > > > said it was out of scope since its really a platform specific
> > > > > implementation detail.
> > > > >
> > > > > If you want a way of laying out your case for xattrs you can try
> > > > > following what I started for Labeled NFS and what Tom continued 
and
> > > > > finally shepherded through the 4.2 adoption process. We have
> > > > > specific use cases in there for security labels. I'm sure you 
can
> > > > > find real concrete examples for the equivalent of the user 
namespace
> > > > > style xattrs from Linux.
> > > >
> > > > I agree that we have to add more to the section on justifying the 
need
> > > for xattr,
> > > > and we will, and invite again members of the WG to contribute to 
this
> > > section,
> > > > like Pranoop and others did. I don't think that the fact that you 
and
> > > Tom had a
> > > > hard time with Labeled NFS should imply that every new feature now
> > > > must have a separate document to justify it. Where is this 
requirement
> > > > coming from? I believe that we need to reach consensus in the WG 
and
> > > > that
> > > should be
> > > > enough to add a new feature. Am I wrong with this assumption?
> > > >
> > > > >
> > > > > Dave
> > > > >
> > > > > _______________________________________________
> > > > > nfsv4 mailing list
> > > > > [e-mail]
> > > > > https://www.ietf.org/mailman/listinfo/nfsv4
> > > > >
> > > >
> > > > _______________________________________________
> > > > nfsv4 mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/nfsv4
> > >
> > >
>