Re: [CDNi] Some questions regarding draft-ma-cdni-metadata

[Kevin J Ma <[e-mail]>]

Ben,

  inline:

> -----Original Message-----
> From: Ben Niven-Jenkins [mailto:[e-mail]]
> Sent: Tuesday, October 18, 2011 8:50 AM
> To: Kevin J Ma
> Cc: [e-mail]
> Subject: Re: [CDNi] Some questions regarding draft-ma-cdni-metadata
> 
> Kevin,
> 
> Please see inline.
> 
> On 17 Oct 2011, at 19:26, Kevin J Ma wrote:
> >> From: [e-mail] [mailto:[e-mail]] On Behalf Of
> >> Ben Niven-Jenkins
> >> I read draft-ma-cdni-metadata but I am struggling a little to
> understand
> >> how it fits into the overall framework of CDNI.
> >>
> >> When I read the draft it appeared to me to be aimed more at a Content
> >> Provider or CDN Provider configuring the authoritative CDN rather that
> for
> >> Metadata exchange between CDNs because of its use of Agents, the fact
> it
> >> keeps all metadata opaque and because a lot of the protocol description
> is
> >> about how to create and modify metadata records which only seems to
> apply
> >> to the authoritative CDN.
> >
> > I attempted to create an extensible representation for metadata and a
> > complete protocol for manipulating metadata, though, not all users of
> > the interface will need all aspects.  Do you feel that the protocol is
> > deficient for non-authoritative CDN use, or that authoritative CDNs do
> > not need the metadata interface?
> 
> I think that we should stick to the scope of work we have been chartered
> to address, in the case of CDNI Metadata that is an interface that "will
> allow the CDNs to exchange content distribution metadata of inter-CDN
> scope".
> 
> If the interface we specify is also usable in other scenarios then that's
> great but I don't think we should be actively looking to design an
> interface that is usable across a multitude of scenarios that are not in
> scope of CDNI because it's a distraction and likely to defocus our
> efforts. Our charter is not to define a new CP->CDN provisioning interface
> (they already exist) so we shouldn't be burning cycles working on things
> that are out of charter.

I personally do not see a difference between uCDN->dCDN and CP->uCDN.
If in supporting uCDN->dCDN you get CP->uCDN, I think that is a bonus.
I do not think we should ignore uCDN->dCDN just because it might be
useful to CPs.  Is the argument that a uCDN should never need to push
metadata to a dCDN and therefore we do not need uCDN->dCDN support?

> >> Agents:
> >> The draft states that "The Agent object contains basic information for
> >> authenticating entities which require access to Metadata." However it
> is
> >> not clear to me how the concept of the Agent object fits within the
> >> framework of CDNI as the details of the different Agents that may be
> able
> >> to access metadata for a Content Service will be specific the to CDN
> >> distributing that CDNI metadata. In other words I can't see why one
> would
> >> need to distribute Agent details between CDNs over CDNI because the
> Agents
> >> that are relevant for an uCDN are likely different to the Agents that a
> >> relevant to a dCDN.
> >
> > It was not intended that agent information be distributed between
> > CDNs, that was what I tried to convery with the text stating:
> >  "Domains and Agents MAY be created as a part of an
> >   off-line business negotiation process or as a part of the CDNI
> >   bootstrapping process."
> > Susan had a similar question about who would be responsible for the
> > creation of agents.  I will clarify that in the next version.
> 
> If Agent information is not supposed to be distributed between CDNs then
> it is not of inter-CDN scope, is out of charter and therefore has no place
> in the context of CDNI IMO.
> 
> >> Later on the draft states:
> >>   The association of each Metadata to an Agent allows different Agents
> >>   to retrieve different Metadata values for a given URI in the given
> >>   Domain.  This is intended to allow CDNs to separate upstream Metadata
> >>   from downstream Metadata (e.g., a uCDN content acquisition URL may
> >>   point to a CP origin, however the content acquisition URL that the
> >>   dCDN retrieves from the uCDN may point at a surrogate in the uCDN).
> >>
> >> This implies that the specification of Agents is really for imposing a
> >> specific implementation on CDNs in how they manage metadata received
> from
> >> a uCDN and how they expose that metadata to dCDNs. As the scope of CDNI
> >> does not cover the internal workings or operation of a particular CDN
> it
> >> would appear to me that imposing a particular internal mechanism for a
> >> single CDN to manage & distribute CDNI metadata is inappropriate for
> the
> >> CDNI WG.
> >
> > The point was that different consumers of the metadata interface may
> > require different responses (or they may not).  Could that be left up
> > to individual implementations?  Perhaps.  I am not sure what internal
> > mechanism this imposes?  Are you saying that CDNI should only define
> > the metadata semantics, and not the storage representation?  My goal
> > was to define a generic representation, that could adapt to any
> > semantics?  The agent is a part of that, though, in the degenerate
> > case of a single agent, nothing is lost?
> 
> We should not be imposing storage representations. We don't need to
> include Agents in CDNI Metadata as they're not distributed between CDNs.
> If CDNs want to expose different Metadata to different dCDNs they can do
> that without the concept of an Agent.
> 
> >> Metadata:
> >> I am a little confused about the desire to keep the actual
> specification
> >> of metadata opaque. In order for a CDN to do anything useful with the
> CDNI
> >> metadata it receives it needs to understand the syntax and semantics of
> >> the metadata it receives, however the draft intentionally keeps both
> the
> >> syntax and semantics for the metadata properties as well as any
> associated
> >> values opaque to any CDN that receives or distributes the associated
> >> metadata and says the associated "functionality SHOULD be agreed upon a
> >> priori by CDN operators, however, these negotiations are beyond the
> scope
> >> of CDNI."
> >>
> >> Of course, the syntax and semantics of the CDNI metadata could be
> agreed
> >> out of band between the operators of the interconnected CDNs but doing
> >> that for all metadata properties that would be needed to meet the
> >> objectives of CDNI does not IMO provide any real advantage over what
> one
> >> could do today by configuring individual services in different CDNs and
> >> using CDNI request routing to redirect requests between those services
> in
> >> each CDN.
> >>
> >> I can see that the meaning of the value for a specific metadata
> property
> >> might need to be opaque because the value may be applied differently by
> >> different CDNs (e.g. a more general "QoS" value that each CDN maps to
> an
> >> appropriate DSCP value depending on how the underlying network has been
> >> configured/deployed) but those are more likely to be exceptions rather
> >> than the general rule and in any case the property itself would still
> need
> >> to be specified & defined, so even with that in mind I'm struggling to
> see
> >> the value in keeping all metadata opaque.
> >
> > My concern with defining all the semantics of metadata and designing a
> > protocol around that, is you typically end up with a less extensible
> > design.  I would not want to have to update the metadata interface rfc
> > just to support a new piece of metadata in the future.  I am not saying
> > that we should not define metadata, I just think it should be separate
> > from the data model and the protocol, so I concentrated on those first.
> > I definitely feel that we should define a base set of metadata that all
> > CDNs should support.  I do not think the draft prevents that.
> 
> We agree on the fact that the CDNI metadata protocol specification doesn't
> need to define all possible properties/semantics for all possible delivery
> protocols and that properties and semantics for different delivery
> protocols can be defined in separate documents.
> 
> The CDNI metadata draft that Grant/David/I authored doesn't preclude
> semantics/properties being defined in other documents (and implies that is
> likely to be the case - see my response to Spencer just now). The base set
> of properties that are described in our draft could easily be broken out
> into a separate document if the consensus of the group is to do so. We
> included the base properties and interface specification in a single
> document so that readers could see how the interface could be constructed
> and applied using a set of common base semantics/properties for HTTP
> delivery supported by deployed CDNs today.
> 
> So if we put aside the question of where to define the actual properties
> and their semantics we're left with the data model and interface/protocol
> itself. So let's compare and contrast some key aspects of flexibility
> between the different proposals.
> 
> Cache-ability & Latency:
> With the CDNI Metadata model we've proposed we think we've managed to
> define a data model that is as simple as possible but no simpler. It has a
> minimum number of data objects to allow objects that are likely to to be
> large or reused to be broken out into separate resources to provide
> efficiency and at the same time objects that may well have different
> cache-ability lifetimes can be broken out into separate resources. The
> interface specification allows inlining so that all related objects for a
> Site can be included in a single request/response so we provide the
> ability to have a simple implementation but also the flexibility to
> optimise the implementation to balance object size and cache-ability
> against number of requests/re-validations required.

The protocol in my draft also supports bulk retrieval, but if there is
a nuance of inlining that is missing, I could certainly look at it.
Though I did not address cacheability, I think it could be applied at
the bulk response level as you describe?

> By keeping your model basically flat you force an implementation to
> repeatedly have to transfer data that is common across a number of
> Hostnames/Domains (e.g. the definition of Locations) and you force the
> cache-ability for all data within a hostname/domain to be the same whether
> it is specific to a hostname/domain or common across multiple
> hostnames/domains. You therefore do not provide any ability for
> implementations to optimise based on their knowledge (or configured
> policies)  of the different types of data that combine to describe the
> CDNI Metadata for a Site/domain/hostname.
> 
> Self-description:
> The links between objects in our model and the representations of the
> different objects in our model are self-describing through the use of
> relationships and media-types within the interface so a client can easily
> determine what data objects are being referenced and what data objects it
> is receiving and use the appropriate processing function to interpret
> their contents. It is easy to add new data objects to our model if
> required by defining new relationships and/or media types. In contrast
> your representations aren't self-describing and so force a tighter
> coupling between the client and server and they require some out of band
> process to ensure that coupling is maintained.

The individual pieces of metadata have a defined type.  If the CDN
supports a given piece of metadata, it will know what the type is.  The
metadata distribution protocol, imo, does not need to know the type
as long as the CDN who receives the metadata understands it?  Knowing
the type is not necessarily enough information to know what to do with
it, the full definition of the metadata is required either way?

> Bootstrapping & Coupling:
> Similarly, with the CDNI Metadata interface/protocol we have proposed, the
> configuration/bootstrapping of the interface is as simple as possible but
> no simpler (a single URI) from which the dCDN can discover the location of
> all the CDNI metadata records they have access to. It also allows the uCDN
> to define the URI format (and hostname it's provided by) as they see fit.
> The uCDN has maximum flexibility to organise the different CDNI metadata
> objects and complete control over how they extend their implementation to
> support additional data objects or how they support mapping different
> dCDNs to different sets of (or versions of) CDNI metadata.
> 
> Your proposal imposes a greater configuration/bootstrapping burden on the
> dCDN as there is no way for it to discover the domains it has access to,
> it needs to know the URI/hostname of the uCDN's Metadata interface and the
> domains to query against that interface. The use of fixed paths in the URI
> that map to the different data object enforces rigidity on the uCDN's CDNI
> Metadata implementation (and tight coupling between the client and server)
> and means that if the uCDN wishes to extend its implementation to support
> additional objects it risks picking paths that may later clash with paths
> defined by newer versions of your protocol.

Your SiteFeed object provides Site information (which is fairly analogous
to domain information).  Domain information could also be put into a feed, 
however, I assume that business processes drive what goes into the SiteFeed.
Is the SiteFeed configured offline, as part of some business process?
If so, then is it better to configure Site information as a feed in
the uCDN, or as domains in the dCDN.  I do not have a strong feeling
either way, but I do not see our schemes as being so different.

thanx.

--  Kevin J. Ma

> Taking the above factors into consideration I believe that our proposed
> data model and interface specification is flexible and extensible enough
> to address the current and future needs of CDNI.
> 
> Ben
> 
> 
>