Re: [CDNi] [E] Cache management interface | Triggers v2

["Mishra, Sanjay" <[e-mail]>]

Hi Alan - Nir and discussed your comments and please see our further
clarifications to your comments


>>    1. For items 1 to 3 are new extensions you are proposing and those
>>    may be just easy to define in a new draft.
>>
>> Sounds good, I'll work on the draft once we finalize the cache management
> spec at SVTA.
>
>
>>    1. For #4, this appears a heavy lift in the current draft as some of
>>    the capabilities such as duration and recurrence and the RFC7265 brings
>>    added complexity for dCDN and some dCDN may not support. It may be best to
>>    leave Timepolicy as is defined in v2 but JCAL policy can be added in the
>>    new draft.
>>
>> I can see that JCAL/iCalendar support can be somewhat of a lift, even
> though there's a number of JCAL libraries out there.
> At the same time, the TimePolicy extension in its current form is a
> partial ad-hoc scheduling implementation, and should be obsoleted by
> RFC7265,
> which is a well-established IETF standard with implementation base.
> Do we want to release CI/T v2 that is obsolete at the time of release?
> [NS/SM] We do not think TimePolicy would be obsoleted. Let the dcdn
> choose what it supports and publish via the FCI (as already defined in the
> draft)
>


> If dCDN doesn't support some advanced JCAL scheduling features like
> recurrence, there's ample room to manage that using eextension error code
> in Error.V2
>
> [NS/SM] This should be managed by a capability object - such a solution
> would be cleaner and mor sustaninable than emitting an error, which is too
> late.
>
We support a capability object to specify which spec can be used with which
> extension. However, the usecase of complex extension is not covered by such
> a general capability.
>
We should probably need to define for JCAL (and every complex extension) an
> additional capability object specifying what part of JCAL it is supported.
> I think that both the new JCAL extension and its capability object should
> be pushed to a separate doc.
>
>>
>>    1.  Items 5 & 6 to generalize playlist to an object list is
>>    reasonable and we can add those in the next revision (v10) early Dec.
>>
>> Great, please let me know what is the best way to add those
> In the spirit of iterative changes, I propose to consider three minor
> features that may belong in the core CI/T v2 spec, rather than the cache
> management extension:
> - add trigger execution priority in Trigger.v2, to allow uCDN to introduce
> urgent triggers
> [NS/SM] While working on the different drafts, we already discussed
> capabilities that allows dependencies between triggers.
>
We concluded that any such capability can be simply defined and added as an
> extension, without forcing the dCDN to support it. We suggest to push it to
> a separate doc.
>


> - add an optional object list to Trigger Status v2; this is to show to
> uCDN how dCDN resolved playlist (aka objectlist) provided to it via
> conteMnt.playlist(objectlist) spec
> [NS/SM] makes sense, also for url patterns. However, wouldnt it
> dramatically enlarge the status object and make the transaction heavier?
>
Addiiotnally note that the extra complexity the dCDN would have to manage
> such a list, especially in a multiple CDN hierarcy topology (e.g. when the
> dCDN has 2 further dCDNs that may provide different replies)
>
Bottom line, we can add it to the respones structure, specifying that it
> should be included only if the "status request" specify so via a flag. However,
> due to the complexity and requirement for concise definitions, maybe to
> add such feature we should add it in a separate doc, with a designated API
> endpoint.
>


>
>
- add an optional object list to Error.v2
> [NS/SM] Again. It might be too heavy and complex.
>
A full description may create an [object * error * cdn-heirarchy] matrix,
> which may be too much.
>
Lets decide how to proceed after resolving which option we agree to take
> wrt the above (status object) question.
>


- update trigger spec objects that use URLs (which is all trigger specs
> except for CCID) to add a field that indicates the type of URL to use -
> published URL vs. cache key URL
>

> If we do that, we can have a clean break between the core CI/T v2 draft on
> hand, and 2 new drafts to be submitted later - cache management trigger
> extensions and
> cache management interface.
> Also, this would be a higher priority than adding JCAL support to v10.
>

[NS/SM] We see 2 options to support that, and need to think about it.
>
Let's take the below example:

    {
         "trigger-subject": "content",
         "generic-trigger-spec-type": "content-playlist",
         "generic-trigger-spec-value": {
             "playlist": "https://www.example.com/hls/title/index.m3u8
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.example.com_hls_title_index.m3u8&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=XniVbishGiO2Ao9hKqSc-hTVIWCi3T-x6GdHR4ZTgoM&m=971EsgFLP9ZLZjNqyHBn_eP79-nLrsreGAuMd4CdJ1bGkZZhLGpP43_nqnMc4Fh-&s=WNpDRf7l_aoJVrQzvMwJZXKtjzmEmn83XzqmqILOIaA&e=>",
             "media-protocol": "hls"
          }
       }



> Option A. We already have a "trigger-subject" proprety ("metadata" /
> "content"). Lets enrich the list of subjects to support the separation of
> URLs type, or add an additional field, e.g. 'trigger-subject-variant"
>

    {
         "trigger-subject": "content",

          "trigger-subject-variant": "published-url",

"generic-trigger-spec-type": "content-playlist",
"generic-trigger-spec-value": { "playlist": "
https://www.example.com/hls/title/index.m3u8
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.example.com_hls_title_index.m3u8&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=XniVbishGiO2Ao9hKqSc-hTVIWCi3T-x6GdHR4ZTgoM&m=971EsgFLP9ZLZjNqyHBn_eP79-nLrsreGAuMd4CdJ1bGkZZhLGpP43_nqnMc4Fh-&s=WNpDRf7l_aoJVrQzvMwJZXKtjzmEmn83XzqmqILOIaA&e=>",
"media-protocol": "hls"

         }
    }


This approach would mean changes in the capability objects.
We currently list the supported (subject, spec, extension) tuples. Instead
we would now have (subject, variants, spec, extension) tuples.

Option B. Define it within the different spec value objects, e.g. with a
"url-type" field
    {

         "trigger-subject": "content",
         "generic-trigger-spec-type": "content-playlist",
         "generic-trigger-spec-value": {
             "playlist": "https://www.example.com/hls/title/index.m3u8
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.example.com_hls_title_index.m3u8&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=XniVbishGiO2Ao9hKqSc-hTVIWCi3T-x6GdHR4ZTgoM&m=971EsgFLP9ZLZjNqyHBn_eP79-nLrsreGAuMd4CdJ1bGkZZhLGpP43_nqnMc4Fh-&s=WNpDRf7l_aoJVrQzvMwJZXKtjzmEmn83XzqmqILOIaA&e=>",

             "url-type": "published",             "media-protocol": "hls"
          }
       }

In this case no capability tuple would be changed. However, we would not be
able to specify whether the different url types are supported or not.

Any preference from your side?


>
>
>    1. For #7, can you provide more information as to what changes to
>    error codes you are considering?
>
> The only error code that we have for now is etimeout, to support TimePolicy
> [SM/NS] Adding desginated error codes whenever an extestion is defined,
> would not scale. There is an "eextension" error code that should be used
> for every extension related error. Furthermore, the error object includes
> a "description" field that can be used - so extension specific error
> information can be placed in the "description" field.
>
When defining a new extension (like a TimePolicy), that may have specific
> erros (e.g. "timeout"), we should also define the required format of the
> error description to allow parsing by the uCDN. So the error would be of
> type "eextension" and the description would be
>
"{subtype: timeout}". Alternatively, we can consider adding a general
> "description-dict" field to the error and utilze it in the same manner
> instead (having a per extension, predefined structre)
>

On Mon, Oct 30, 2023 at 10:22 AM Alan Arolovitch <[e-mail]> wrote:

> Sanjay, Nir,
> Thanks, see my comments below
>
> Best regards
> Alan
>
> On Sun, Oct 29, 2023 at 5:09 PM Mishra, Sanjay <[e-mail]>
> wrote:
>
>> Hi Alan - Nir and I discussed your email below, and recommend the
>> following:
>>
>>
>>    1. For items 1 to 3 are new extensions you are proposing and those
>>    may be just easy to define in a new draft.
>>
>> Sounds good, I'll work on the draft once we finalize the cache management
> spec at SVTA.
>
>
>>    1. For #4, this appears a heavy lift in the current draft as some of
>>    the capabilities such as duration and recurrence and the RFC7265 brings
>>    added complexity for dCDN and some dCDN may not support. It may be best to
>>    leave Timepolicy as is defined in v2 but JCAL policy can be added in the
>>    new draft.
>>
>> I can see that JCAL/iCalendar support can be somewhat of a lift, even
> though there's a number of JCAL libraries out there.
> At the same time, the TimePolicy extension in its current form is a
> partial ad-hoc scheduling implementation, and should be obsoleted by
> RFC7265,
> which is a well-established IETF standard with implementation base.
> Do we want to release CI/T v2 that is obsolete at the time of release?
> If dCDN doesn't support some advanced JCAL scheduling features like
> recurrence, there's ample room to manage that using eextension error code
> in Error.V2
>
>>
>>    1.  Items 5 & 6 to generalize playlist to an object list is
>>    reasonable and we can add those in the next revision (v10) early Dec.
>>
>> Great, please let me know what is the best way to add those
> In the spirit of iterative changes, I propose to consider three minor
> features that may belong in the core CI/T v2 spec, rather than the cache
> management extension:
> - add trigger execution priority in Trigger.v2, to allow uCDN to introduce
> urgent triggers
> - add an optional object list to Trigger Status v2; this is to show to
> uCDN how dCDN resolved playlist (aka objectlist) provided to it via
> content.playlist(objectlist) spec
> - add an optional object list to Error.v2
> - update trigger spec objects that use URLs (which is all trigger specs
> except for CCID) to add a field that indicates the type of URL to use -
> published URL vs. cache key URL
>
> If we do that, we can have a clean break between the core CI/T v2 draft on
> hand, and 2 new drafts to be submitted later - cache management trigger
> extensions and
> cache management interface.
> Also, this would be a higher priority than adding JCAL support to v10.
>
>    1. For #7, can you provide more information as to what changes to
>    error codes you are considering?
>
> The only error code that we have for now is etimeout, to support TimePolicy
>
>> Thanks
>> Sanjay/NIr
>>
>> On Thu, Oct 26, 2023 at 1:26 PM Alan Arolovitch <[e-mail]> wrote:
>>
>>>
>>> Hello
>>>
>>> I'm leading the cache management interface project at SVTA. The scope is
>>> to define a full-fledged CDNi-compliant cache management interface as part
>>> of the open caching architecture
>>> We are heading towards completion of the initial spec draft, and I would
>>> like to share my thoughts on how to best bring this work into CDNi.
>>> The current high-level scope includes:
>>> - Trigger-based cache data operations that rely on and extend CDNi
>>> triggers v2 as currently defined in CDNI CI/T 2nd edition
>>> - Additional CDNi metadata objects allowing uCDN to assign cache objects
>>> in configuration hierarchy to cache buckets; tag cache objects for use in
>>> triggers via content.ccid trigger type; indicate cache object priority
>>> - Cache bucket management API
>>>
>>> The current thinking is to split the work into two documents - one that
>>> specifies extensions to triggers v2, and the other that pertains to cache
>>> management specific metadata and related API.
>>>
>>> If so, would it make sense to incorporate the triggers v2 changes into
>>> the current draft - all or some of them, or introduce them as a separate
>>> document?
>>>
>>> The proposed triggers v2 change scope is as follows:
>>> 1. New trigger extension PrepositionPolicy, per paragraph 8 of CDNI CI/T
>>> v2, governing prepositioning policies like retry, patrial retrieval,
>>> preposition methods, concurrency and bandwidth, and use of CDNi metadata
>>> 2. New trigger extension PurgePolicy, governing purge semantics
>>> 3. New trigger extension CommonPolicy, governing trigger execution
>>> priority and type of URLs referenced in triggers (published vs. cache-key
>>> URLs)
>>> 4. Changes to trigger extension TimePolicy (8.2 in CDNi CI/T v2), adding
>>> trigger execution scheduling, based on JSON-based iCAL [RFC7265], allowing
>>> specifying start, stop, recurrence, duration, timezone etc.
>>> 5. Changes of new content.playlist trigger spec(7.5.1 in CDNi CI/T v2)
>>> to content.objectlist spec, in conjunction of introducing new types of
>>> object list, in addition to hls, dash and mss types per 7.5.2 in CDNi CI/T
>>> v2, allowing specifying lists of objects for trigger in JSON-encoded and
>>> plain text formats
>>> 6. Changes to Trigger Status Resource v2 (6.1.3 in CDNi CI/T v2) to
>>> specify objectlists that were derived from content.objectlist triigger spec
>>> 7. Changes to error codes in Error v2 to support the new extension
>>> functionality
>>>
>>> One possible option is to incorporate changes to existing objects (items
>>> 4-7) in the current draft, and defer the new trigger extensions (1-3) to a
>>> separate document.
>>>
>>> Sanjay, Kevin,
>>> I would appreciate some time in the open mic section in Prague to talk
>>> about this
>>>
>>> Kind regards
>>> Alan
>>>
>>>
>>> _______________________________________________
>>> CDNi mailing list
>>> [e-mail]
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_cdni&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=XniVbishGiO2Ao9hKqSc-hTVIWCi3T-x6GdHR4ZTgoM&m=klLmU7WL8P8gyd8m8z7YB4NCt4EbVkWgNY8xAAHx_jHe9zZVF9x85ykXTw7n3KgG&s=9VNsXLtJaYQ0fxZWKYHdXOx9VLDAIzhrpd1OxV4QrR8&e=
>>>
>>
>