Re: [nfsv4] request for xattr as a WG item

[Marc Eshel <[e-mail]>]

Hi Alex,

Please read the following draft and than see what questions you have left.

https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01

Thanks, Marc.



From:   "McDonald, Alex" <[e-mail]>
To:     Spencer Shepler <[e-mail]>, Marc 
Eshel/Almaden/IBM@IBMUS, "[e-mail]" <[e-mail]>
Date:   03/24/2015 10:16 AM
Subject:        RE: [nfsv4] request for xattr as a WG item



This is feedback based not on implementation possibilities or issues but 
on the role of extended attributes and metadata. 
 
There is a great deal of talk about extended attributes, when what appears 
to be meant is data that is 
 
a.       retrievable by key
b.      bound to a root (the base data or value)
 
Given that the proposal would appear to require keys and values that are 
not fixed or limited in size, I have several questions that aren’t 
answered here (and I apologise if there is some other part of the proposal 
that I’ve not seen or read that explains this).
 
a.       Security. Do the metadata/xattrs have the same security as the 
base data? 
b.      Searchability. Can users expect to search for keys or keys and 
values, and how would the protocol support it?
c.       Placement. Does the proposal imply colocation of metadata/xattrs 
with the base data, or are there requirements where they might  be 
divided? 
d.      Updates and locking. Is the metadata/xattrs supported by locking 
or some “eventual consistency” model? 
e.      Versioning. Is it anticipated to provide support for this?
f.        Namespace. Do metadata/xattrs require internal and/or external 
namespace(s) and extension to filenames? Will metatdat/xattrs be 
addressable via URIs?
g.       Streams. Is it possible to treat the metadata/xattrs as a file in 
its own right?
 
Thanks.
 
Alex McDonald
Standards & Industry Associations Group
CTO Office
NetApp
[phone] Mobile Phone
[e-mail]
twitter: @alextangent
Follow us: 
 
 
From: nfsv4 [mailto:[e-mail]] On Behalf Of Spencer Shepler
Sent: 24 March 2015 16:02
To: Marc Eshel; [e-mail]
Subject: Re: [nfsv4] request for xattr as a WG item
 
 
Any feedback for Marc?  I would like to close this week on the decision to 
move this to WG work….
 
From: Marc Eshel [mailto:[e-mail]] 
Sent: Thursday, March 12, 2015 1:31 PM
To: Spencer Shepler
Cc: Marc Eshel; [e-mail]
Subject: RE: [nfsv4] request for xattr as a WG item
 
This a sample usage of xattr that was requested to help reaching consensus 
for the need of xattr and move the draft to a WG I-D 
Thanks, Marc. 


Background: 
While extended attributes ( a.k.a. xaatr or EA) are around in most 
filesystems for quite some time, standard applications that uses them are 
not that common. 
It seems that one of the main inhibitors for that is lack of support in 
standard file sharing protocols ( NFS for example). 
This document goal is to describe use cases in which customers are using 
xattr ( over proprietary protocols or CIFS/SMB) and would probably do so 
over NFS if this feature would be available. 

Extended attributed in Digital media: 
One of the main use cases  that comes to mind when discussing xattr as a 
mean to store extended information as part of a file metadata is in the 
digital media industry. 
One of the biggest challenges in this industry is managing content, which 
goes through various platforms and phases during the processing workflow. 
If in the past, media content was mostly produced by legacy film based 
sources, mostly of professional grade – now days, media content may be 
produced by different authors ( professional and amateurs) using different 
tools ( cell phones, security cams, professional grade equipment etc.) 
following different standards and platforms ( Windows based, Linux, 
Android, Osx, IOS etc.), needs to be processed using various tools ( video 
editing, image processing etc.) and shared with the via various  platforms 
( web, TV etc.) to various end devices ( different resolution, quality 
etc.) 
Thus, keeping content metadata in a standard form through the lifetime of 
the content is highly required – but in reality, lack of standards is 
making this type of content management a nightmare, especially using open 
tools (1) 

Example use case – digital video: 
 Let's looks on this example workflow: 
 1.     Video is being captured on a fixed camera into its embedded Linux 
box 
 2.     Video is edited using Video editing software on Mac OSX box 
 3.     Video is reviewed by the reporter  on his Windows desktop 
 4.     Video is transmitted on air – to consumers TVs using proprietary 
system 
 5.     Video is shared on the TV station web site 
  
So, lets look again in the above example and try to add some more metadata 
context: 
--------------------------------------------------------------------------- 

| Workflow stage                | attribute/tags              | comments   
                  | 
--------------------------------------------------------------------------- 

| Capture                        |Location, time              |   | 
--------------------------------------------------------------------------- 

| Editing                        | Editor name, change tracking,        |  
                             | 
|                                        | content comments, generation   
|                                | 
--------------------------------------------------------------------------- 

| Review                                | Speaker notes, content tags    | 
                               | 
--------------------------------------------------------------------------- 

| On air transmission| Quality/resolution tags(HD/SD)        | Multiple 
version        | 
--------------------------------------------------------------------------- 

| Web sharing                | Copyright/trademark,additional        | 
Multiple versions        | 
|                                        | quality/resolution tags  |      
           | 
--------------------------------------------------------------------------- 


  
As can be seen above, not only that the content itself is being 
transformed and enhanced during its lifetime – its metadata is enriched 
and needs to be accessed by multiple applications on multiple platforms. 
Today, for the most part, a 3rd party application is required in order to 
manage the content metadata and/or proprietary filesystems that can store 
those attributed with the file 
 – potentially on all the platforms. And even with those, there might be 
compatibility issues (2) 

Current solutions: 
So, it seems what some customers are doing is using SMB/CIFS for the 
purpose of sharing the data between platforms. The drawback of that is 
that SMB/CIFS is somewhat not working well. 
Attempts to use NFS ( as it looks like the best logical solution) – don't 
work ( and even breaks the apps completely) as it don't support xattr (3) 

Conclusion 
In today's world, managing the large variety of content becomes a 
challenging task. Extended attributes can ( and do ) assist with that – 
but unfortunately they cannot be used with NFS due to lack of support. 

Other examples: 
There are other cases in which xattr starts to gain traction, including: 
 1. HDFS: 
http://mapredit.blogspot.co.il/2014/07/xattr-are-coming-to-hdfs.html 
 2. Openmeta: https://code.google.com/p/openmeta/ 


References: 
(1) 
http://www.strategyand.pwc.com/global/home/what-we-think/reports-white-papers/article-display/media-workflow-digital-era 

(2) https://discussions.apple.com/message/19411219#19411219 
(3)http://superuser.com/questions/427581/osx-extended-attributes-and-nfs 





From:        Spencer Shepler <[e-mail]> 
To:        Marc Eshel/Almaden/IBM@IBMUS 
Cc:        "[e-mail]" <[e-mail]> 
Date:        10/08/2014 05:54 PM 
Subject:        RE: [nfsv4] request for xattr as a WG item 





Marc,

I have not observed consensus as of yet -- I see positive comments and 
engagement and with use-case clarity we should be able to move forward.

The use-case item does not need to land in an I-D if you are looking for 
minimal overhead.  A email to the WG with a draft of what would land in 
the I-D would be sufficient to get feedback and to demonstrate consensus.

BTW: just to state the obvious, when an item becomes a WG I-D does not 
guarantee forward movement...

Spencer

> -----Original Message-----
> From: Marc Eshel [mailto:[e-mail]]
> Sent: Wednesday, October 8, 2014 3:29 PM
> To: Spencer Shepler
> Cc: Dave Quigley; Marc Eshel; [e-mail]; nfsv4
> Subject: RE: [nfsv4] request for xattr as a WG item
> 
> Hi Spencer,
> 
> We have enough proposed changes from the feedback in the last few days 
that
> a new update to the draft is needed, but after working on it for more 
than a
> year and watching several rounds of similar discussions at different 
times on
> the list and at IETF, I would like to hear some assurance from the 
chairs that
> the WG will take it up as an item. We take it as an action item to 
update the
> draft, but this is not a personal mission, and needs support from the
> community. I just want confirmation that this is not a waste of our 
time.
> 
> Thanks, Marc.
> 
> 
> 
> From:   Spencer Shepler <[e-mail]>
> To:     Marc Eshel/Almaden/IBM@IBMUS, Dave Quigley
> <[e-mail]>,
> Cc:     "[e-mail]" <[e-mail]>, nfsv4 <[e-mail]
>
> Date:   10/08/2014 02:25 PM
> Subject:        RE: [nfsv4] request for xattr as a WG item
> 
> 
> 
> 
> Marc,
> 
> Do you have an idea of when an update to the I-D may be available?
> 
> Spencer
> 
> > -----Original Message-----
> > From: nfsv4 [mailto:[e-mail]] On Behalf Of Marc Eshel
> > Sent: Tuesday, October 7, 2014 6:33 PM
> > To: Dave Quigley
> > Cc: [e-mail]; nfsv4
> > Subject: Re: [nfsv4] request for xattr as a WG item
> >
> > "nfsv4" <[e-mail]> wrote on 10/07/2014 06:08:15 PM:
> >
> > > From: Dave Quigley <[e-mail]>
> > > To: [e-mail],
> > > Date: 10/07/2014 06:08 PM
> > > Subject: Re: [nfsv4] request for xattr as a WG item Sent by: "nfsv4"
> > > <[e-mail]>
> > >
> > > On 10/6/2014 6:11 PM, Marc Eshel wrote:
> > > >
> > > > We added a section to the draft based on your request to explain
> > > > why
> > it is
> > > > needed, but you are still not convinced. I think that you are
> > > > looking
> > for
> > > > a killer application that will prove the need for xattr and I
> > > > don't
> > know
> > > > of one. all I know is what we added to the draft and what
> > > > customers
> > tell
> > > > us, but it is not enough for you. So I don't know how to convince
> > > > you, others on the WG seem to agree that it is needed, I assume
> > > > based on
> > their
> > > > knowledge and experience. I can not give you a name of an
> > > > application
> > that
> > > > a customer is running because I don't know the name and even if I
> > > > did
> > it
> > > > would mean nothing, all I can say that for example one application
> > > > is
> > used
> > > > to tag files using xattr for future migration, it is like a policy
> > > > for file movement among the different storage levels. Maybe you
> > > > should
> > talk to
> > > > your customers and see if they needed it.
> > > >
> > >
> > >
> > > The thing is that just having a section justifying the feature is
> > > not enough. I learned that when I started Labeled NFS. I started
> > > with a requirements document. I then was asked to a use case
> > > document, an impact study, a requirements document, and finally the
> > > actual specification which got merged into 4.2. Compared to the
> > > varied semantics of xattr implementations on different systems
> > > security labels were easy. We punted on syntax and semantics and
> > > abstracted that out into a separate document. We punted on
> > > establishing a security model and
> >
> > > said it was out of scope since its really a platform specific
> > > implementation detail.
> > >
> > > If you want a way of laying out your case for xattrs you can try
> > > following what I started for Labeled NFS and what Tom continued and
> > > finally shepherded through the 4.2 adoption process. We have
> > > specific use cases in there for security labels. I'm sure you can
> > > find real concrete examples for the equivalent of the user namespace
> > > style xattrs from Linux.
> >
> > I agree that we have to add more to the section on justifying the need
> for xattr,
> > and we will, and invite again members of the WG to contribute to this
> section,
> > like Pranoop and others did. I don't think that the fact that you and
> Tom had a
> > hard time with Labeled NFS should imply that every new feature now
> > must have a separate document to justify it. Where is this requirement
> > coming from? I believe that we need to reach consensus in the WG and
> > that
> should be
> > enough to add a new feature. Am I wrong with this assumption?
> >
> > >
> > > Dave
> > >
> > > _______________________________________________
> > > nfsv4 mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/nfsv4
> > >
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/nfsv4
> 
>