Re: [vwrap] Inventory, Asset Metadata and Privacy (was: [wvrap] Simulation consistency)

[Morgaine <[e-mail]>]

I'd better tighten up one paragraph I wrote, because it could be interpreted
as "all metadata is public", which wasn't intended.  Here's an improved
version:


"Access to the *content* of a restricted asset may be achieved by first
acquiring a capability for that asset.  A capability needs to be acquired
only when that asset's metadata indicates that the content is restricted.
Access to the *metadata* of an asset is never restricted, for anyone, once
an asset has been placed in-world.  Until an asset is placed in-world, its
metadata address is not known to anyone except the owner of the asset, and
is unguessable.  When an asset service restricts access to the content of a
particular asset, then different policies may be applied to agents who hold
assets in inventory and agents who merely render assets worn by others in
the region.  The asset service holds authority over the asset, and has the
power to discriminate."


And further on access controls, a point which we haven't yet discussed:


"If access to the content of a restricted asset is attempted without using a
capability, the access attempt will be rejected by the asset service because
insufficient credentials were provided.  Consequently, simply removing the *
Restricted* property from a restricted asset's metadata in order to avoid
requesting a capability cannot bypass required access controls."


Those are actually reasonable paragraphs for a draft document describing the
design model of VWRAP asset services, assuming there is agreement. :-)


Morgaine.




===========================


On Mon, Apr 11, 2011 at 4:25 AM, Morgaine <[e-mail]>wrote:

>
>
> Sorry, I was ambiguous.  What I should have said was "Only access to the
> content of assets may be restricted by access controls, and only when it is
> appropriate to do so (eg. never for open content).  Access to the metadata
> of assets is never restricted, for anyone.  When access controls restrict
> access to the content of a particular asset, then different policies may be
> applied to agents who hold assets in inventory and agents who merely render
> assets worn by others in the region.  The asset service holds authority over
> the asset, and has the power to discriminate."
>
>
> <cut>



On Mon, Apr 11, 2011 at 4:25 AM, Morgaine <[e-mail]>wrote:

> Excellent response, Boroondas, full of detailed analysis! :-)
>
> I won't respond to everything in-line because it would be mostly a long
> sequence of "Yes!" and "Good idea!", but I'll expand in a couple of areas
> where your reply makes me think that I should have been clearer, and also
> where I prefer your suggestions to mine.
>
> What's more, I'll adopt your suggested nomenclature straight away:
>
> *asset == (metadata + content)*
>
> I like that!  "Metadata + data" has never felt right, because everything is
> "data". :-)
>
>
>
>
> On Sun, Apr 10, 2011 at 10:27 PM, Boroondas Gupte <
> [e-mail]> wrote:
>
>>
>>
>> > On 04/10/2011 03:11 PM, Morgaine wrote:
> >> The metadata has to be available to remote parties in the same way as
> the data itself is, and the best way of handling this is to make metadata a
> separate item normally stored in the same asset service as the data.
>
> As far as I know, this differs slightly from the concepts/architecture in
> SL. This might be a good thing, but because it is almost, but not
> completely, the same, we should be aware of these differences: In SL, there
> are the concept of assets (which are what I called 'content' above) and the
> concept of inventory items, referencing these assets and holding additional
> information such as permissions, name, etc. These inventory items themselves
> however are *not* considered assets in SL.
>
> So far, that's just a difference in nomenclature, I guess. But the SL
> inventory items are also handled differently from SL assets: They aren't
> stored (and served to the region) by the asset server, like assets are, but
> instead by the inventory service, which also serves the inventory tree. In
> fact, at least in the viewer's data model, the folders making up that tree
> are just special inventory items that do not reference assets but reference
> other inventory items.
>
> One of the drawbacks of how SL draws the line between inventory (-items)
> and assets is that it's probably difficult to implement something like
> VWR-2427 <https://jira.secondlife.com/browse/VWR-2427>. Treating the
> item-metadata as a type of assets as you seem to suggest might allow to
> better exploit the commonality that both inventory items and some data
> assets can refer to other assets.
>
> (Note that I'm not really an expert on SL's internal architecture, and most
> of the above is based on hear-say and some viewer code comments I remember.
> Please correct me if I'm wrong somewhere.)
>
>
> I'm not an expert on SL's architecture either, but our discussions in the
> AWG over 3+ years suggest that your analysis above is correct.  We are
> actually sticking fairly closely to the *conceptual* model used in SL (not
> for compatibility, but because it is a reasonable model), but we are
> departing from SL's *physical* model in some significant ways to gain
> better functional properties.
>
> According to the very rough consensus in AW Groupies (:P), in SL only the
> *content* is stored in *asset storage*, whereas *metadata* is stored in a
> *relational database* (and is mutable).  In the design that we are
> discussing here for VWRAP asset services, both *content* and *metadata *are
> held in asset services, and both types of data are immutable.  Our "updates"
> are achieved simply by referring to new metadata.
>
> SL handles its metadata specially and caches it in an inventory cache
> separately from content, whereas we store and cache them uniformly in both
> asset services and caches.  Our inventories merely organize our cached
> metadata into a hierarchical *view*.  We're building on past experience to
> improve functionality and performance and scalability (and elegance too!),
> which is quite natural.
>
> The "information about assets" stored at the leaves of inventories in
> SL-compatible clients is conceptually very similar to the metadata stored in
> our proposed asset services.  Even the content referencing model is similar,
> in the sense that capabilities are used in SL to obtain the UUIDs of
> textures for example, and these UUIDs are then stored in the metadata held
> in inventory items.
>
> In other words, UUIDs held in SL inventory leaves are equivalent to our *direct
> addresses* (URIs) held in metadata.  In both cases, once you have the UUID
> or *direct address*, you no longer need a capability to obtain the item.
> The cap's only function is to control acquisition of the UUID, which is the
> actual address of the content.  Our content URIs just extend the concept of
> an address to external asset services.
>
>
> >> The metadata item will naturally reference the corresponding data item
> (which is an N:1 relationship when hash-based addressing is used), and in
> some cases can reference more than one data item --- for example, the
> metadata of a mesh will commonly reference not only the graphical data
> required for rendering in the client but also the collision mesh or bounding
> box data required by the region.
>
> > Wouldn't it make more sense to keep the metadata-item-to-data-item
> relation strictly N:1, and have the mesh data item have a N:N relation to
> it's components? (Analogous to SL gestures.) While the item type should be
> part of the metadata, I don't think the metadata's structure should depend
> on the item type. After all, the structure of inodes of a file system also
> doesn't depend on the file formats of the referenced files.
>
>>
>>
> Yes!  That makes more sense.  I withdraw my N:M suggestion. :-)  Let's
> stick with N:1 and then be more clever with content if needed later.  This
> is also in keeping with my cost engineering mantra, namely "The burden of X
> should be borne only by those who need X".  There is no sense complicating
> metadata to make it N:M when the vast majority of its use cases are N:1.
>
>
>
>  >> As a final point about metadata versus data, it should be mentioned
> that generally only the data ever has access restrictions placed on it,
>
> > No restrictions for whom? Just for the user whose inventory it is (but
> then he/she has to authenticate somehow) or no restrictions for anyone?
>
>
> Sorry, I was ambiguous.  What I should have said was "Only access to the
> content of assets may be restricted by access controls, and only when it is
> appropriate to do so (eg. never for open content).  Access to the metadata
> of assets is never restricted, for anyone.  When access controls restrict
> access to the content of a particular asset, then different policies may be
> applied to agents who hold assets in inventory and agents who merely render
> assets worn by others in the region.  The asset service holds authority over
> the asset, and has the power to discriminate."
>
>
>
> > ("inventory" here refers to just the tree, if I understood you
> correctly). I guess this is also the case for other users, except where I
> have given them explicit permission to view parts of my inventory (if the
> protocol was capable of that).
>
>
> I do not envisage anyone ever having access to an agent's inventory, other
> than the agent who owns the inventory (inventories are client-side after
> all, despite also being saved server-side).  Only the metadata that an
> inventory may be referencing can ever be seen by others (for example, when a
> region provides the references), but never the hierarchical structure.  This
> is similar to how SL/Opensim works:  you can inspect the metadata of an
> asset in the scene and see that it is owned by user X, but you have no
> access to X's inventory.
>
>
>
> >> I am going to assume that nobody wants that [poor functionality and lag]
> and hence, that metadata is never accessed through capabilities.
>
> > Now here I'm beginning to wonder, what data all is part of the tree (that
> others aren't able to access) and what is instead part of the item-metadata,
> (a fraction of) which needs to be broadcasted to other users, as it contains
> the references to the assets those other users have to fetch to render my
> avatar correctly. For example, is the item name part of that metadata or
> part of the tree? It obviously isn't needed by others, it's merely a
> reminder for myself of what that item is.
>
>
> Just in case I wasn't clear before, I'll underline that inventory trees are
> entirely unrelated to the process by which others gain access to assets
> (they use metadata which links to content, not inventories).  The only
> reason why I mentioned inventories at all is because they too happen to
> point to metadata.  As to the question of what is in inventories that isn't
> in metadata, the answer is "almost nothing" other than the hierarchical
> skeleton of named folders.  I guess it's possible to have local symlinks,
> and even to modify the metadata locally if desired, for example to change
> the local name of an item, but those are just frills.  Generally, inventory
> is merely metadata organized locally into a tree.
>
>
>
> > If the item name for worn stuff *is* broadcasted together with the other
> relevant metadata, that's OK as long as the user is aware of it, but it *
> does* have some consequences: In that case, I probably wouldn't want to
> name a wearable "The dress that this stupid guy bought me, but that I like
> to wear anyway" or "Those shoes that my secret lover Example Resident gave
> me" but something more innocent, so that I wouldn't upset that other user
> and wouldn't leak that Example Resident was my secret lover.
>
> Haha. :-)  Well it's already the case in SL that the metadata is
> inspectable, so people are already used to the fact that how they name
> objects and what they write in object descriptions is visible publicly,
> nothing new there.
>
>
>
> > There might be other metadata that's useful to the users themselves but
> that they might want to withhold from region owners and other users. Imagine
> for example information about where and when you acquired an item, which
> allows you to efficiently search for that item when you forgot the item name
> but still remember how you got it. However, if that info is broadcasted for
> worn items and you wear enough different items, everyone around you has a
> partial trace of your virtual travels.
>
>
> Yes, good ideas for purely local enhancements to inventories. :-)  I'm sure
> that inventories can be extended in many useful ways to build upon their
> primary role of organizing and referencing metadata.  That could be a very
> interesting area.  Anyone with dozens of thousands of objects in inventory
> can appreciate very well the need for some new thinking to help us manage
> the object explosion. :-)
>
>
> Morgaine.
>
>
>
>
> ======================
>
> On Sun, Apr 10, 2011 at 10:27 PM, Boroondas Gupte <
> [e-mail]> wrote:
>
>>  Hi Morgaine
>>
>> The distinction between metadata and 'content' and storing them in
>> separately acquirable chunks is a good and quite important idea, I think.
>> More generally, I can think of several reasons to split a monolithic data
>> entity E into several separate ones, connected by referencing each other or
>> by being referenced by a common meta-entity which replaces the previous
>> monolithic entity:
>>
>>    1. Several other entities are expected to have some parts in common
>>    with E, without having other parts in common with it.
>>    2. Some parties only need some parts of E, but not other parts.
>>     3. Some parties are only allowed access to some parts of E, but not
>>    to other parts.
>>
>> (You might note the similarity between reason (1) and rules used for
>> database normalization.)
>>
>> If the entities are immutable (which assets should be according to
>> previous discussion<http://www.ietf.org/mail-archive/web/vwrap/current/threads.html#00796>),
>> (1) can not only occur between unrelated items but will also be very common
>> for items that result from changing a previous item. Some examples how this
>> applies specifically to the metadata-content separation (assuming a similar
>> inventory UI as we have in SL):
>>
>>    - You rez an object from inventory, resize it, and take it back into
>>    inventory (which creates a new item there).
>>       - Same metadata (permissions, name(?), owner)
>>          - (What about a 'last edited' date, though? SL doesn't seem to
>>          store one, but some VWRAP systems might want to keep that info.)
>>           - Different content
>>    - You rename an item in-inventory
>>       - Different metadata (maybe ... see below)
>>        - Same content
>>    - You change next-owner permissions on an item, then give it to
>>    someone
>>       - Different metadata
>>       - Same content
>>
>> Reason (2) off course occurs when browsing your inventory. If you just
>> want to e.g. read the names of your items, you don't have to fetch all the
>> associated content data.
>>
>> Off course, there are more potential separations that fit one or several
>> of these three reasons than just the metadata-content one. In SL, having
>> objects being composed of prims, having prims reference textures for their
>> faces instead of having the texture data lumped into their own data, or
>> gestures being composed of steps some of which can reference animation and
>> sound assets are other examples of such separations.
>>
>> On 04/10/2011 03:11 PM, Morgaine wrote:
>>
>> It is worth noting that while inventories are mostly a client-side
>> implementation detail which can be changed unilaterally, the *metadata*held by inventories is a very important and separable information type in an
>> architecture that uses 3rd party asset services, so it needs detailed
>> examination.
>>
>> The metadata has to be available to remote parties in the same way as the
>> data itself is, and the best way of handling this is to make metadata a
>> separate item normally stored in the same asset service as the data.
>>
>> As far as I know, this differs slightly from the concepts/architecture in
>> SL. This might be a good thing, but because it is almost, but not
>> completely, the same, we should be aware of these differences: In SL, there
>> are the concept of assets (which are what I called 'content' above) and the
>> concept of inventory items, referencing these assets and holding additional
>> information such as permissions, name, etc. These inventory items themselves
>> however are *not* considered assets in SL.
>>
>> So far, that's just a difference in nomenclature, I guess. But the SL
>> inventory items are also handled differently from SL assets: They aren't
>> stored (and served to the region) by the asset server, like assets are, but
>> instead by the inventory service, which also serves the inventory tree. In
>> fact, at least in the viewer's data model, the folders making up that tree
>> are just special inventory items that do not reference assets but reference
>> other inventory items.
>>
>> One of the drawbacks of how SL draws the line between inventory (-items)
>> and assets is that it's probably difficult to implement something like
>> VWR-2427 <https://jira.secondlife.com/browse/VWR-2427>. Treating the
>> item-metadata as a type of assets as you seem to suggest might allow to
>> better exploit the commonality that both inventory items and some data
>> assets can refer to other assets.
>>
>> (Note that I'm not really an expert on SL's internal architecture, and
>> most of the above is based on hear-say and some viewer code comments I
>> remember. Please correct me if I'm wrong somewhere.)
>>
>> The metadata item will naturally reference the corresponding data item
>> (which is an N:1 relationship when hash-based addressing is used), and in
>> some cases can reference more than one data item --- for example, the
>> metadata of a mesh will commonly reference not only the graphical data
>> required for rendering in the client but also the collision mesh or bounding
>> box data required by the region.
>>
>> Wouldn't it make more sense to keep the metadata-item-to-data-item
>> relation strictly N:1, and have the mesh data item have a N:N relation to
>> it's components? (Analogous to SL gestures.) While the item type should be
>> part of the metadata, I don't think the metadata's structure should depend
>> on the item type. After all, the structure of inodes of a file system also
>> doesn't depend on the file formats of the referenced files.
>>
>> These two types of data are separate because it would be inefficient to
>> require clients or regions to download data that they do not use.  Our
>> "asset" singleton now turns into a metadata + data pair.
>>
>> Yep.
>>
>>  As a final point about metadata versus data, it should be mentioned that
>> generally only the data ever has access restrictions placed on it,
>>
>> No restrictions for whom? Just for the user whose inventory it is (but
>> then he/she has to authenticate somehow) or no restrictions for anyone?
>>
>>  because placing restrictions on metadata leads to unsearchable
>> inventories for which the technical term is "annoying as hell", or more
>> seriously, poorly functionality and yet another source of lag.
>>
>> I agree everyone should be able to access their own inventory tree,
>> including the item-metadata at its leaf nodes. For the tree part, you later
>> write
>>
>> Regions don't have access to your inventory [...]
>>
>> ("inventory" here refers to just the tree, if I understood you correctly).
>> I guess this is also the case for other users, except where I have given
>> them explicit permission to view parts of my inventory (if the protocol was
>> capable of that).
>>
>> I am going to assume that nobody wants that [poor functionality and lag]
>> and hence, that metadata is never accessed through capabilities.
>>
>> Now here I'm beginning to wonder, what data all is part of the tree (that
>> others aren't able to access) and what is instead part of the item-metadata,
>> (a fraction of) which needs to be broadcasted to other users, as it contains
>> the references to the assets those other users have to fetch to render my
>> avatar correctly. For example, is the item name part of that metadata or
>> part of the tree? It obviously isn't needed by others, it's merely a
>> reminder for myself of what that item is.
>>
>> If the item name for worn stuff *is* broadcasted together with the other
>> relevant metadata, that's OK as long as the user is aware of it, but it *
>> does* have some consequences: In that case, I probably wouldn't want to
>> name a wearable "The dress that this stupid guy bought me, but that I like
>> to wear anyway" or "Those shoes that my secret lover Example Resident gave
>> me" but something more innocent, so that I wouldn't upset that other user
>> and wouldn't leak that Example Resident was my secret lover.
>>
>> There might be other metadata that's useful to the users themselves but
>> that they might want to withhold from region owners and other users. Imagine
>> for example information about where and when you acquired an item, which
>> allows you to efficiently search for that item when you forgot the item name
>> but still remember how you got it. However, if that info is broadcasted for
>> worn items and you wear enough different items, everyone around you has a
>> partial trace of your virtual travels.
>>
>> So we can either split the metadata asset according to reason (3) above
>> into several assets with different access policies. Or we just lump metadata
>> we want to keep private together with the tree while only having metadata we
>> don't mind sharing in the metadata assets.
>>
>> Cheers,
>> Boroondas
>>
>> _______________________________________________
>> vwrap mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/vwrap
>>
>>
>