Re: [CDNi] New Metadata Interface Draft (draft-cjlmw-cdni-metadata-00)

[Ben Niven-Jenkins <[e-mail]>]

Hi Kevin,

Thanks for reading the draft, some good questions. My opinion inline.


On 16 Jul 2012, at 21:40, Kevin J Ma wrote:

> Hi Matt,
>  
>   A couple of initial comments/questions about the merged draft:
>  
>   - I do not understand the need for a heirarchical separation of the
>     Acquisition and Delivery objects?  Why is the property "key" not
>     sufficient for designating the function of the metadata?

This is a good question and something we discussed at length amongst the authors but probably don't really talk about in the draft.

We discussed a lot the trade offs between embedding all CDNI metadata properties together versus splitting some out into different object that could be independently cached/retrieved.

We decided not to try to optimise too early so we took an approach of allowing any object to be embedded or independently addressable. This leads to the data model being slightly more convoluted and it can certainly be optimised a bit IMO. But we decided on an approach where we first had a stable/agreed model with the WG and then we could revisit it see what optimisations make sense.
>  
>     Though it could be argued that these are logically different
>     functions, the extra layer seems unnecessary, and introduces some
>     complexity to the inheritance model?
>  
>     With inheritance-based override, can a lower level PathMetadata
>     object override just an Auth List, or does it have to replace the
>     entire Delivery object?

This is an area where we need to flesh out the specification and not one we've fully fleshed out yet.

My current thinking is to only have to specify those properties which you want to override compared to the parent PathMetadata object. 

>  Does it have to place the Auth List
>     inside a lower level Delivery object, or can it just define a new
>     Auth List with no Delivery object wrapper?

Currently yes but when we revisit the data model to look for optimisations this is a good candidate.

>  Is there a concept of
>     leaf vs. branch objects for override purposes?

I'm not exactly sure what you mean by leaf vs branch

>  
>     In general, if there are an arbitrary number of levels below each
>     object, and an arbitrary number of levels of PathMetadata which
>     may contain objects with arbitrary levels of metadata which may
>     override previous levels, is there a need to check that duplicate
>     Metadata do not exist within a given subtree, and/or should that
>     restriction be stated explicity?

I don't think there's a need to check for duplicates explicitly as they represent different content items/paths/hosts but a uCDN may want to do so as part of optimising it's representation of the metadata but that's an implementation decision.

>  
>   - I see that the PathMetadata defines some restrictions to what it
>     may contain, however, this does not seem like a scalable method of
>     managing object restrictions.

I disagree. It is an editorial device to avoid repeating definitions for the same properties with identical text in multiple places in the document.

>  As we move forward and more vendor
>     specific metadata are defined, managing metadata collisions using
>     the such a method seems complex, in that the CDNI Metadata RFC
>     could need to be continuously updated to add more restrictions.
>     It would be nice if there was a general set of rules for how and
>     where to define new metadata, rather than creating highly
>     customized objects which have complex modification restrictions?

We're not defining complex modification restrictions we're ruling out of scope those properties which simply do not make sense to include in a PathMetadata object.

How we handle vendor extensions is a separate issue, that I will admit needs some more work and the ideas in the current draft need refinement.

However both HostMetadata & PathMetadata objects would allow vendor extensions and it is up to the specification for those extensions to decide whether the appropriate place to place their extension is within the HostMetdata, PathMetadata, or both objects.

>  
>   - I assume that the inheritance model requires that PathMatch
>     objects must match the path of all PathMatch objects above it in
>     the hierarchy?  i.e., I cannot add a PathMatch for "/bar/baz"
>     inside a PathMetadata whose PathMatch was for "/foo"?  It is
>     probably worth stating explicitly?

Yes. In your example there would not be a URL a client could request that would ever match both path patterns and so the CDNI Metdata associated with the /foo path would never get applied. I have no problem stating that more explicitly in the next version.
 
>  
>     How are ambiguous patterns (e.g., "/*/foo/*") handled?  Are there
>     defined restrictions on the PathMatch objects to guarantee that
>     all lookups will be deterministic?

IMO The PathMatches are ordered and the first match wins, so in the case of ambiguous/overlapping patterns the uCDN needs to ensure it has ordered the PathMatches appropriately to get the desired deterministic behaviour.
 
>  
>   - In the ACLRule object, if you can only have either an allow or a
>     deny Property but not both, and the ACLRule list can only have
>     either Location or TimeWindow properties but not both, it is not
>     clear that much is gained by combining them into the same ACL and
>     ACLRule objects?  In the future, if new types of ACLs are added,
>     would we modify these existing objects (and rev the CDNI Metadata
>     RFC), or just add separate object definitions?

I think this is an example of where in trying to explain the concept of ACLs earlier on in the document we carried that abstract model maybe a little too far into the encoding. We should re-look at that part to see if we can clean it up along the lines you propose.
 
>  
>     The reuse of object definitions, in this case, does not seem to
>     buy us anything?  Defining separate LocationACL/LocationACLRule
>     objects and TimeWindowACL/TimeWindowACLRule objects, where
>     allow/deny is just an attribute/property would result in
>     essentially the same implementation, but without the
>     interdependendies, which would simplify future revisions?
>  
>   - In the Property definitions, I assume that the "key" is the term
>     that follows the "Property:" tag, and that the "Description:" and
>     "Type:" indented below describe the "value"?

Correct.

>  
>     It was not clear to me what the "Mandatory:" tag implied?
>     Mandatory to specify the Property in every object?  Mandatory to
>     implement support for the Property?  or Mandatory to enforce the
>     Property?  It might be useful to explicitly define Mandatory?

Mandatory to specify the property in every object.

My thinking is that for a given delivery protocol there will be a mandatory set of properties/capabilities a dCDN is expected to have. If the dCDN cannot support the required "flavour" of the delivery protocol, it cannot perform the delivery (and would advertise that as part of its capabilities etc.). 

Generally in the CDNI Metadata specification, if a property is mandatory to specify in an object it is because it is essential for the delivery of that delivery protocol.

For a given delivery protocol there would need to be a separate specification (IMO) that specified the mandatory to implement/understand CDNI Metadata properties in order for a dCDN to claim "compliance" with a given delivery protocol.

>  
>   - In the HostMatch object, the hostname Property is defined as a
>     String?  Should there be Pattern support for hostnames?

I'd need to think about that a bit more first but I think there might be some value to that.

>  Also,
>     does hostnames support both IP and DNS values?

I'm not sure of the use case for IP values as clients connecting to CDNs aren't typically given a URL with an IP address as a hostname.

>   Also, does the
>     hostname support a URI prefix that should be matched as part of
>     the hostname, and not be included in the PathMatch?

No, the hostname is essentially the HTTP Host: header. All URL prefixes are via PathMetadata.

>  
>   - For Lists of objects, I did not see a specific ordering defined?
>     Is there a way to enforce ordering?

Lists are ordered. In JSON they would be an array with N entries, with index[0] being first and index[N-1] being last.


>  
>   - With respect to extensibility, can an "x-" object be added to any
>     other object, or are there restrictions on where a new custom
>     object may be added into the hierarchy?

It's up to whoever specifies the "x-" property to define which objects it applies to and what restrictions apply to it.

I think this is an area of the draft that isn't fully fleshed out and needs more thought and your input is welcome.

My current thinking is to split extensions clearly into "mandatory" and "optional" categories which could for example be expressed by having a property of "mandatory" which contained a dictionary of 'mandatory' extensions and likewise for optional extensions.

The basic rule would be if there is a property in the "mandatory" category that you don't understand then you cannot perform the content delivery. You can still perform the delivery if there are "optional" extensions you don't understand.

But like I say it's an area I think we definitely need to do some more thinking, refinement etc of.

>  
>     Can I add a new ACL type?  Can I add a new pattern matching option
>     to the HostMatch and/or PathMatch objects (fundamentally changing
>     the way hosts and paths are matched)?  Can I add new types of
>     location algorithms (e.g., GPS coordinates or zip code) to the
>     Location object?  Or is it intended more for just creating a new
>     hierarchy under the Delivery object?

IMO you can potentially do all those things. I think if you had lots of them as mandatory extensions you might be better off minting a vendor specific delivery protocol identifier to give dCDNs a heads-up that they needed to support a lot of vendor/extension-specific capabilities, but there are certainly other ways to do that too (depending how the capabilities advertisement interface evolves).

>  
>   - With respect to the "ignorable" Property, does that affect the
>     entire parent object to which it is added?  If it is a Property
>     itself (rather than an attribute of a Property), then does that
>     require a wrapper object around the actual Property and the
>     "ingnorable" Property, for all new Properties?  Should it be a
>     standard attribute like Mandatory?
>  
>   - Is there a way for the client to request metadata for a given URI,
>     rather than recursively checking and requesting PathMetadata?
>     Doesn't this lead to alot of lookups (even local cache lookups)
>     for each request, to find all of the applicable PathMetadata?
>     Are there optimizations for local cache lookup?

This partly goes back to optimising the data model once we have it stable. But also in reality I don't expect the layers of hierarchical metadata to extend to that many and if a dCDN is being selected one would hope that it is doing a fairly decent amount of delivery for that domain/uCDN and so it would need most of that portion of the metadata tree anyway.

So the initial lookup from the "root" of the CDNI Metadata tree to the PathMetadata for the domain/site is non optimal but the results are highly cacheable and so any penalty (assuming the dCDN doesn't prefetch the top couple of tiers of objects anyway) is only on first retrieval (from that point there's nothing to stop the dCDN pre-validating cache freshness for objects to make sure it always has a fresh version in it's cache).


>  
>   Note: I have also uploaded an updated version of my draft:
>         http://tools.ietf.org/html/draft-ma-cdni-metadata-03 

Thanks. I'll give it a read when I get a chance.

Ben

>  
> thanx.
>  
> --  Kevin J. Ma
>  
>  
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Matt Caulfield (mcaulfie)
> Sent: Monday, July 09, 2012 3:45 PM
> To: [e-mail]
> Cc: Ben Niven-Jenkins ([e-mail])
> Subject: [CDNi] New Metadata Interface Draft (draft-cjlmw-cdni-metadata-00)
>  
> Hi everyone,
>  
> As we announced in April, a group of us (Ben Niven-Jenkins, Kent Leung, Rob Murray, Grant Watson, and I) have been working on a merge of two of the CDNI metadata interface proposals:
> -       draft-caulfield-cdni-metadata-core-00
> -       draft-jenkins-cdni-metadata-00
>  
> The merged draft has now been submitted: http://tools.ietf.org/html/draft-cjlmw-cdni-metadata-00
>  
> We will be presenting on this draft in Vancouver. Any feedback on the list is also appreciated.
>  
> Thanks,
> Matt
>  
>  
>                                                                  
> _______________________________________________
> CDNi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/cdni