Re: [CDNi] Some questions regarding draft-ma-cdni-metadata

[Ben Niven-Jenkins <[e-mail]>]

Kevin,

Please see inline.

On 17 Oct 2011, at 19:26, Kevin J Ma wrote:
>> From: [e-mail] [mailto:[e-mail]] On Behalf Of
>> Ben Niven-Jenkins
>> I read draft-ma-cdni-metadata but I am struggling a little to understand
>> how it fits into the overall framework of CDNI.
>> 
>> When I read the draft it appeared to me to be aimed more at a Content
>> Provider or CDN Provider configuring the authoritative CDN rather that for
>> Metadata exchange between CDNs because of its use of Agents, the fact it
>> keeps all metadata opaque and because a lot of the protocol description is
>> about how to create and modify metadata records which only seems to apply
>> to the authoritative CDN.
> 
> I attempted to create an extensible representation for metadata and a
> complete protocol for manipulating metadata, though, not all users of
> the interface will need all aspects.  Do you feel that the protocol is
> deficient for non-authoritative CDN use, or that authoritative CDNs do
> not need the metadata interface?

I think that we should stick to the scope of work we have been chartered to address, in the case of CDNI Metadata that is an interface that "will 
allow the CDNs to exchange content distribution metadata of inter-CDN 
scope".

If the interface we specify is also usable in other scenarios then that's great but I don't think we should be actively looking to design an interface that is usable across a multitude of scenarios that are not in scope of CDNI because it's a distraction and likely to defocus our efforts. Our charter is not to define a new CP->CDN provisioning interface (they already exist) so we shouldn't be burning cycles working on things that are out of charter.

>> Agents:
>> The draft states that "The Agent object contains basic information for
>> authenticating entities which require access to Metadata." However it is
>> not clear to me how the concept of the Agent object fits within the
>> framework of CDNI as the details of the different Agents that may be able
>> to access metadata for a Content Service will be specific the to CDN
>> distributing that CDNI metadata. In other words I can't see why one would
>> need to distribute Agent details between CDNs over CDNI because the Agents
>> that are relevant for an uCDN are likely different to the Agents that a
>> relevant to a dCDN.
> 
> It was not intended that agent information be distributed between
> CDNs, that was what I tried to convery with the text stating: 
>  "Domains and Agents MAY be created as a part of an
>   off-line business negotiation process or as a part of the CDNI
>   bootstrapping process."
> Susan had a similar question about who would be responsible for the
> creation of agents.  I will clarify that in the next version. 

If Agent information is not supposed to be distributed between CDNs then it is not of inter-CDN scope, is out of charter and therefore has no place in the context of CDNI IMO.

>> Later on the draft states:
>>   The association of each Metadata to an Agent allows different Agents
>>   to retrieve different Metadata values for a given URI in the given
>>   Domain.  This is intended to allow CDNs to separate upstream Metadata
>>   from downstream Metadata (e.g., a uCDN content acquisition URL may
>>   point to a CP origin, however the content acquisition URL that the
>>   dCDN retrieves from the uCDN may point at a surrogate in the uCDN).
>> 
>> This implies that the specification of Agents is really for imposing a
>> specific implementation on CDNs in how they manage metadata received from
>> a uCDN and how they expose that metadata to dCDNs. As the scope of CDNI
>> does not cover the internal workings or operation of a particular CDN it
>> would appear to me that imposing a particular internal mechanism for a
>> single CDN to manage & distribute CDNI metadata is inappropriate for the
>> CDNI WG.
> 
> The point was that different consumers of the metadata interface may
> require different responses (or they may not).  Could that be left up
> to individual implementations?  Perhaps.  I am not sure what internal
> mechanism this imposes?  Are you saying that CDNI should only define
> the metadata semantics, and not the storage representation?  My goal
> was to define a generic representation, that could adapt to any 
> semantics?  The agent is a part of that, though, in the degenerate
> case of a single agent, nothing is lost?

We should not be imposing storage representations. We don't need to include Agents in CDNI Metadata as they're not distributed between CDNs. If CDNs want to expose different Metadata to different dCDNs they can do that without the concept of an Agent.

>> Metadata:
>> I am a little confused about the desire to keep the actual specification
>> of metadata opaque. In order for a CDN to do anything useful with the CDNI
>> metadata it receives it needs to understand the syntax and semantics of
>> the metadata it receives, however the draft intentionally keeps both the
>> syntax and semantics for the metadata properties as well as any associated
>> values opaque to any CDN that receives or distributes the associated
>> metadata and says the associated "functionality SHOULD be agreed upon a
>> priori by CDN operators, however, these negotiations are beyond the scope
>> of CDNI."
>> 
>> Of course, the syntax and semantics of the CDNI metadata could be agreed
>> out of band between the operators of the interconnected CDNs but doing
>> that for all metadata properties that would be needed to meet the
>> objectives of CDNI does not IMO provide any real advantage over what one
>> could do today by configuring individual services in different CDNs and
>> using CDNI request routing to redirect requests between those services in
>> each CDN.
>> 
>> I can see that the meaning of the value for a specific metadata property
>> might need to be opaque because the value may be applied differently by
>> different CDNs (e.g. a more general "QoS" value that each CDN maps to an
>> appropriate DSCP value depending on how the underlying network has been
>> configured/deployed) but those are more likely to be exceptions rather
>> than the general rule and in any case the property itself would still need
>> to be specified & defined, so even with that in mind I'm struggling to see
>> the value in keeping all metadata opaque.
> 
> My concern with defining all the semantics of metadata and designing a
> protocol around that, is you typically end up with a less extensible
> design.  I would not want to have to update the metadata interface rfc
> just to support a new piece of metadata in the future.  I am not saying
> that we should not define metadata, I just think it should be separate
> from the data model and the protocol, so I concentrated on those first.
> I definitely feel that we should define a base set of metadata that all
> CDNs should support.  I do not think the draft prevents that.

We agree on the fact that the CDNI metadata protocol specification doesn't need to define all possible properties/semantics for all possible delivery protocols and that properties and semantics for different delivery protocols can be defined in separate documents.

The CDNI metadata draft that Grant/David/I authored doesn't preclude semantics/properties being defined in other documents (and implies that is likely to be the case - see my response to Spencer just now). The base set of properties that are described in our draft could easily be broken out into a separate document if the consensus of the group is to do so. We included the base properties and interface specification in a single document so that readers could see how the interface could be constructed and applied using a set of common base semantics/properties for HTTP delivery supported by deployed CDNs today.

So if we put aside the question of where to define the actual properties and their semantics we're left with the data model and interface/protocol itself. So let's compare and contrast some key aspects of flexibility between the different proposals.

Cache-ability & Latency:
With the CDNI Metadata model we've proposed we think we've managed to define a data model that is as simple as possible but no simpler. It has a minimum number of data objects to allow objects that are likely to to be large or reused to be broken out into separate resources to provide efficiency and at the same time objects that may well have different cache-ability lifetimes can be broken out into separate resources. The interface specification allows inlining so that all related objects for a Site can be included in a single request/response so we provide the ability to have a simple implementation but also the flexibility to optimise the implementation to balance object size and cache-ability against number of requests/re-validations required.

By keeping your model basically flat you force an implementation to repeatedly have to transfer data that is common across a number of Hostnames/Domains (e.g. the definition of Locations) and you force the cache-ability for all data within a hostname/domain to be the same whether it is specific to a hostname/domain or common across multiple hostnames/domains. You therefore do not provide any ability for implementations to optimise based on their knowledge (or configured policies)  of the different types of data that combine to describe the CDNI Metadata for a Site/domain/hostname.

Self-description:
The links between objects in our model and the representations of the different objects in our model are self-describing through the use of relationships and media-types within the interface so a client can easily determine what data objects are being referenced and what data objects it is receiving and use the appropriate processing function to interpret their contents. It is easy to add new data objects to our model if required by defining new relationships and/or media types. In contrast your representations aren't self-describing and so force a tighter coupling between the client and server and they require some out of band process to ensure that coupling is maintained.

Bootstrapping & Coupling:
Similarly, with the CDNI Metadata interface/protocol we have proposed, the configuration/bootstrapping of the interface is as simple as possible but no simpler (a single URI) from which the dCDN can discover the location of all the CDNI metadata records they have access to. It also allows the uCDN to define the URI format (and hostname it's provided by) as they see fit. The uCDN has maximum flexibility to organise the different CDNI metadata objects and complete control over how they extend their implementation to support additional data objects or how they support mapping different dCDNs to different sets of (or versions of) CDNI metadata.

Your proposal imposes a greater configuration/bootstrapping burden on the dCDN as there is no way for it to discover the domains it has access to, it needs to know the URI/hostname of the uCDN's Metadata interface and the domains to query against that interface. The use of fixed paths in the URI that map to the different data object enforces rigidity on the uCDN's CDNI Metadata implementation (and tight coupling between the client and server) and means that if the uCDN wishes to extend its implementation to support additional objects it risks picking paths that may later clash with paths defined by newer versions of your protocol.

Taking the above factors into consideration I believe that our proposed data model and interface specification is flexible and extensible enough to address the current and future needs of CDNI.

Ben