Re: [CDNi] New Metadata Interface Draft (draft-cjlmw-cdni-metadata-00)

["Matt Caulfield (mcaulfie)" <[e-mail]>]

Hi Kevin,

Your feedback is very helpful.

My responses inline.

Thanks,
Matt

-----Original Message-----
From: Kevin J Ma [mailto:[e-mail]] 
Sent: Wednesday, July 18, 2012 3:10 PM
To: Matt Caulfield (mcaulfie); [e-mail]
Cc: Ben Niven-Jenkins ([e-mail])
Subject: RE: New Metadata Interface Draft (draft-cjlmw-cdni-metadata-00)

Hi Matt,

  thanx for the reponse.  comments inline:

> From: Matt Caulfield (mcaulfie) [mailto:[e-mail]]
> Sent: Tuesday, July 17, 2012 4:02 PM
> To: Kevin J Ma; [e-mail]
> Cc: Ben Niven-Jenkins ([e-mail])
> Subject: RE: New Metadata Interface Draft 
> (draft-cjlmw-cdni-metadata-00)
> 
> Hi Kevin,
> 
> Thank you for the comments, please see my responses inline below.
> 
> Regards,
> Matt
> 
> From: Kevin J Ma [mailto:[e-mail]]
> Sent: Monday, July 16, 2012 4:40 PM
> To: Matt Caulfield (mcaulfie); [e-mail]
> Cc: Ben Niven-Jenkins ([e-mail])
> Subject: RE: New Metadata Interface Draft 
> (draft-cjlmw-cdni-metadata-00)
> 
> Hi Matt,
> 
>   A couple of initial comments/questions about the merged draft:
> 
>   - I do not understand the need for a heirarchical separation of the
>     Acquisition and Delivery objects?  Why is the property "key" not
>     sufficient for designating the function of the metadata?
> [Matt>] The separation was created to categorize metadata properties. 
> It's not essential. It may be useful if one server in a CDN is 
> delivery a piece of content and another server is acquiring. The 
> acquisition server could limit itself to looking at the acquisition 
> metadata while the delivery server could limit itself to looking at 
> the delivery metadata. Would you suggest removing this distinction?

It raises question in my mind about how override works (see response to Ben).
[Matt>] I think I understand your confusion. The best way to think about the inheritance rules is not by what overrides what. Instead, imagine that the dCDN needs to know the value of parameter X. The dCDN has already found the branch of HostMetadata and PathMetadata which apply to the request. To find the value of parameter X, the dCDN first looks for that parameter in the leaf PathMetadata. If the value is not found, it looks in the parent, and if it's not found, looks in the parent's parent, all the way up. If the parameter value is not found, it takes on the default value. 

>     Though it could be argued that these are logically different
>     functions, the extra layer seems unnecessary, and introduces some
>     complexity to the inheritance model?
> [Matt>] I don't think that the extra layer introduces any additional 
> complexity. In our model, objects can contain objects which can 
> contain objects, the delivery/acquisition objects aren't a special case.
> 
>     With inheritance-based override, can a lower level PathMetadata
>     object override just an Auth List, or does it have to replace the
>     entire Delivery object?  Does it have to place the Auth List
>     inside a lower level Delivery object, or can it just define a new
>     Auth List with no Delivery object wrapper?  Is there a concept of
>     leaf vs. branch objects for override purposes?
> [Matt>] Any object at any level can be overridden. So yes, just the 
> Auth list could be overridden, there is no need to override the entire 
> delivery object. Yes, the schema must be preserved - so to override 
> the auth list it would be wrapped by a delivery object. No, there is 
> no concept of leaf vs. branch. Based on your comment, I think we could 
> spend more time on the description of inheritance.
> 
>     In general, if there are an arbitrary number of levels below each
>     object, and an arbitrary number of levels of PathMetadata which
>     may contain objects with arbitrary levels of metadata which may
>     override previous levels, is there a need to check that duplicate
>     Metadata do not exist within a given subtree, and/or should that
>     restriction be stated explicity?
> [Matt>] I'm not sure what you mean by duplicate metadata. Each 
> PathMetadata object defines a bunch of metadata properties, each one 
> is unique. Each PathMetadata object either inherits or overrides the 
> metadata properties of its parent.

I guess I was thinking more about scope, not across multiple PathMetadata objects, but within a single one, e.g.:

Delivery has an auth property.  If someone add a new x-azuki property to the Delivery property and the x-azuki property has its own auth property.
Is there a conflict?

Presumably the delivery/x-azuki/auth property is distinct from the delivery/auth property?  And the precedence of the two auth properties would be defined by the x-azuki/auth property?  If there was also a delivery/x-cisco/auth property and a delivery/x-velocix/auth it gets a little more complex.  Should there be restrictions on naming and/or overriding existing metadata object functions, or is it just left up to the implementors to do the right thing?
 [Matt>] I think Ben supplied a good answer to this question.

>   - For Lists of objects, I did not see a specific ordering defined?
>     Is there a way to enforce ordering?
> [Matt>] I'm not sure I understand your question. All lists are ordered.
> Can you clarify?

I do not see any text in the draft which defines what a "List" is, or how they are ordered?  JSON arrays have inherent ordering, but XML lists do not.  Also, JSON and XML may only be an on-the-wire encoding.  The data model should specify an order from which to generate the JSON/XML?
[Matt>] Noted, we will need to add a description of Lists to the next revision in the data model section. In XML encoding, we'll need to add an attribute which provides the list index (this was an oversight).

>   - With respect to extensibility, can an "x-" object be added to any
>     other object, or are there restrictions on where a new custom
>     object may be added into the hierarchy?
> [Matt>] An "x-" object can be added anywhere.
> 
>     Can I add a new ACL type?  Can I add a new pattern matching option
>     to the HostMatch and/or PathMatch objects (fundamentally changing
>     the way hosts and paths are matched)?  Can I add new types of
>     location algorithms (e.g., GPS coordinates or zip code) to the
>     Location object?  Or is it intended more for just creating a new
>     hierarchy under the Delivery object?
> [Matt>] It is intended for adding new metadata properties, however it 
> could be used for changing the structure of the metadata tree or 
> changing how we do path matches as you suggest.
> 
>   - With respect to the "ignorable" Property, does that affect the
>     entire parent object to which it is added?  If it is a Property
>     itself (rather than an attribute of a Property), then does that
>     require a wrapper object around the actual Property and the
>     "ingnorable" Property, for all new Properties?  Should it be a
>     standard attribute like Mandatory?
> [Matt>] Yes, the ignorable property affects the entire object and all 
> the objects it contains. The "ignorable" property can be thought of as 
> an attribute. In the XML encoding, it would likely be an attribute 
> (and not an element). In JSON, yes, it can lead to wrapper object.

So, would "ignorable" affect all children as well?  Or only the objects in the current level, not including other objects referenced through a PathMatch object?
[Matt>] "ignorable" affects all children as well. Will need to clarify this in the next revision when we improve the extensibility section.

>   - Is there a way for the client to request metadata for a given URI,
>     rather than recursively checking and requesting PathMetadata?
>     Doesn't this lead to alot of lookups (even local cache lookups)
>     for each request, to find all of the applicable PathMetadata?
>     Are there optimizations for local cache lookup?
> [Matt>] No, there's no way to get the metadata for a specific URL 
> without traversing the tree. This approach enforces cacheability. As 
> for local lookups, it's up to the CDN how it store metadata 
> internally. The local lookup speed is independent of the metadata interface speed.

I'm not sure that cacheability is really affected.  I understand the desire to be able to individually reference and store objects.  The recursive tree structure does not really enhance that.  Having all the PathMatch objects in a flat single layer structure would not change cacheability (or increase the storage requirements), but it might make it easier to natively support lookup for given URIs?
 
[Matt>] With respect to cacheability, I was referring to the idea of requesting metadata for a given URI. Assume that we form a request for a metadata object by putting the content request URL in the query string to the metadata server. If this is the approach, then I don't think that HTTP caching can help us. 

[Matt>] With respect to flat vs tall tree, I think you're right - cacheability is not affected. However, I think that the multi-level tree provides better space efficiency for a few reasons. 1) With the flat tree there could be redundancy between siblings. In a multi-level model, any redundant metadata would be kept in a single parent node. 2) In a flat tree, the root node would be much larger than the multi-level tree. 3) In a multi-level tree, the cache TTL of each node could be proportional to its tree level (the root could have a long TTL).

Though compacted representations could be generated for faster local lookups, the un-compacted representation is still needed to be able to determine how to refresh metadata or get more specific (additional PathMatch) metadata.
[Matt>] This information could be stored in whatever format is most reasonable to the dCDN. It need not be "un-compacted". 

thanx.

--  Kevin J. Ma

>   Note: I have also uploaded an updated version of my draft:
>         http://tools.ietf.org/html/draft-ma-cdni-metadata-03
> [Matt>] Thanks, I will check it out.
> 
> thanx.
> 
> --  Kevin J. Ma
> 
> 
> From: [e-mail] [mailto:[e-mail]] On Behalf 
> Of Matt Caulfield (mcaulfie)
> Sent: Monday, July 09, 2012 3:45 PM
> To: [e-mail]
> Cc: Ben Niven-Jenkins ([e-mail])
> Subject: [CDNi] New Metadata Interface Draft 
> (draft-cjlmw-cdni-metadata-
> 00)
> 
> Hi everyone,
> 
> As we announced in April, a group of us (Ben Niven-Jenkins, Kent 
> Leung, Rob Murray, Grant Watson, and I) have been working on a merge 
> of two of the CDNI metadata interface proposals:
> - draft-caulfield-cdni-metadata-core-00
> - draft-jenkins-cdni-metadata-00
> 
> The merged draft has now been submitted: 
> http://tools.ietf.org/html/draft-
> cjlmw-cdni-metadata-00
> 
> We will be presenting on this draft in Vancouver. Any feedback on the 
> list is also appreciated.
> 
> Thanks,
> Matt
> 
> 
>