Re: [icnrg] I just submitted an update to my ICN QoSArch draft

["David R. Oran" <[e-mail]>]

Excellent, thanks. I’ll wait to put in an update until I hear back 
from Dirk & Börje about whether they agree with my proposed next steps. 
I’ll be sure this gets in before heading to RG last call.

On 4 Dec 2019, at 9:34, Luca Muscariello wrote:

> On Sat, Nov 30, 2019 at 2:53 PM David R. Oran <[e-mail]> 
> wrote:
>
>> Thanks Luca, very helpful.
>> Couple of followup questions:
>>
>>    1.
>>
>>    Despite the caveat that I’m obviously oversimplifying, you point 
>> out
>>    that I’m not being “100% accurate” in my new text about 
>> bandwidth
>>    allocation. Could you perhaps suggest some (hopefully 
>> terse/simple) wording
>>    that would improve the accuracy?
>>
>>
> As short as possible
>
> "There is per-Interest/Data state at every hop of the path and 
> therefore
> outstanding Interests
> provide information that can be used to optimize resource allocation 
> for
> data returning on the inverse path, such as
> bandwidth sharing, prioritization and overload control".
>
>
>>
>>    1.
>>
>>    The background giving theory justification for the assertions 
>> about
>>    ICN congestion control advantages might be really useful for some 
>> readers.
>>    Might you suggest one or two references I could add?
>>
>>
> This is one example about how and why outstanding interests are 
> related to
> network resources optimization.
> Full disclosure, I'm a co-author of this paper.
>
> https://doi.org/10.1016/j.comnet.2016.09.012
>
>
> Luca
>
>
>
>>
>> Best,
>> DaveO.
>>
>> On 29 Nov 2019, at 10:43, Luca Muscariello wrote:
>>
>>
>>
>> On Thu, Nov 28, 2019 at 5:12 PM David R. Oran <[e-mail]>
>> wrote:
>>
>>> On 17 Nov 2019, at 19:27, Thomas C. Schmidt wrote:
>>>
>>> Hi Dave,
>>>
>>> as promised, here comes some feedback on the QoSArch draft.
>>>
>>> Thanks a ton!
>>>
>>> Typos have already been spotted by Anil, so just a general remark on
>>> capitalization: C/consumers and P/producers, also I/interest etc. 
>>> are not
>>> always treated uniformly.
>>>
>>> See my replies to Anil, - I think I fixed all the typos he pointed 
>>> out.
>>> For the consistent capitalization, I chose to have consumer and 
>>> producer
>>> consistently lower case, and Interest capitalized. -03 will have 
>>> these and
>>> the other changes below based on your comments.
>>>
>>> The following is in chronological order:
>>>
>>> Section 3.1:
>>> "There is per-Interest/Data state at every hop of
>>> the path and therefore for each outstanding Interest, bandwidth for
>>> data returning on the inverse path can be allocated."
>>>
>>> IMO this sounds simpler than it is, even if the size of data chunks 
>>> is
>>> known. Traffic is returning with an unknown delay. More 
>>> specifically,
>>> traffic is governed by a stochastic rate equation, and stochastic 
>>> calculus
>>> tells us that delay and delay variation are first order terms. At 
>>> least in
>>> the current Internet, the delay variation attains the same order of
>>> magnitude as the delay itself, thus making actual traffic return 
>>> rates hard
>>> to predict.
>>>
>>> Well, I am certainly oversimplifying the situation. I was hoping 
>>> that by
>>> just saying “allocated” I could avoid people like you paying 
>>> attention and
>>> pointing out that the stochastic nature of RTTs makes accurate 
>>> allocation
>>> difficult. For open-loop sources like IP the delay and delay 
>>> variation are
>>> in fact of similar magnitude, which of course means deterministic
>>> pre-allocation will get things wrong nearly all the time. The 
>>> situation is
>>> somewhat better for request-response closed-loop protocols like ICN, 
>>> but
>>> your basic point still holds.
>>>
>>> I suspect there will be other readers with congestion control clue 
>>> who
>>> will pick up on this, so I definitely ought to add something to the
>>> document. I struggled with just how much, and came up with the 
>>> following,
>>> which I will add to the end of section 3.1:
>>>
>>> “Given the stochastic nature of round trip times, and the ubiquity 
>>> of
>>> wireless links and encapsulation tunnels with variable bandwidth, a 
>>> simple
>>> scheme that admits interests only based on a time-invariant estimate 
>>> of the
>>> returning link bandwidth will perform poorly. However, two 
>>> characteristics
>>> of NDN and CCNx-like protocols can substantially help to improve the
>>> accuracy and responsiveness of the bandwidth allocation:
>>>
>>>    1.
>>>
>>>    RTT is bounded by the Interest lifetime, which puts an upper 
>>> bound on
>>>    the RTT uncertainty for any given Interest/Data exchange. If 
>>> Interest
>>>    lifetimes are kept reasonably short (a few RTTs) the allocations 
>>> do not
>>>    have to deal with an arbitrarily long tail. One could in fact do 
>>> a
>>>    deterministic allocation on this basis, but the result would be 
>>> highly
>>>    pessimistic. Nevertheless, having a cut-off does improve the 
>>> performance of
>>>    an optimistic allocation scheme.
>>>    2.
>>>
>>>    Returning Data packets can be congestion marked by an ECN-like
>>>    marking scheme if the inverse link starts experiencing long queue 
>>> occupancy
>>>    or other congestion indication. Unlike TCP/IP, where the rate 
>>> adjustment
>>>    can only be done end-to-end, this feedback is usable immediately 
>>> by the
>>>    downstream ICN forwarder and the Interest shaping rate lowered 
>>> after a
>>>    single link RTT. This may allow less pessimistic rate adjustment 
>>> schemes
>>>    than the AIMD with .5 multiplier that is used on TCP/IP networks. 
>>> It also
>>>    allows the rate adjustments to be spread more accurately among 
>>> the
>>>    Interest/Data flows traversing a link sending congestion 
>>> signals.”
>>>
>>>
>> The instantaneous number of pending interests (N_t) is the integral 
>> of the
>> difference between interest(I_t) and data rates (D_t):
>> N_t = \Int_[0,t] max(0, I_u - D_t)du.
>>
>> This is a virtual queue measured at a given node.
>> As these rates can be associated naturally on a per-packet (1:1) 
>> basis and
>> aggregated and disaggregated as needed (per interface, per prefix 
>> etc)
>> several virtual queues can be computed. These are the equivalence 
>> classes
>> according to Dave's definition.
>>
>> These virtual queues can be associated naturally to the Lagrange
>> multipliers associated to the congested links
>> in the network and therefore they are equivalent to ECN signals, 
>> which, in
>> the case of TCP, can also be associated
>> to congested links as they are fundamentally associated to the 
>> capacity
>> constraints in a global network optimization problem.
>>
>> There is previous work where these concepts have been formally 
>> defined,
>> that I can report if needed.
>>
>> Even if delays follow a stochastic process that does not mean they 
>> cannot
>> be used in practice.
>> Stochastic is not a synonym of unpredictable. In the case of a 
>> virtual
>> queues (as defined above),
>> the integral formulation is a lot more robust than delay measurements
>> (first or second order statistics)
>> and could be used in theory to optimize resource allocation.
>>
>> If Dave's text may be not 100% accurate, I think the idea is valid 
>> and
>> would be good to have it
>> spelled out entirely in the document, with maybe reference with the
>> theoretical concepts.
>> This is the fundamental underlying advantage, with a very solid
>> theoretical explanation
>> of local flow balance and why it is formally superior to IP to 
>> control
>> congestion.
>>
>> Nonetheless, I agree with Thomas when he says, "this sounds simpler 
>> than
>> it is"!
>>
>>
>>
>>
>>>
>>>
>>> What do you think? This will go in -03 unless you have further 
>>> comments
>>> soon.
>>>
>>> Section 4, Table 2: I guess this still needs some love ;)
>>>
>>> xml2rfc gives me little to no control over text table formatting. 
>>> I’m
>>> going to leave this for now since I’m not motivated to do a lot of 
>>> work to
>>> defeat this stupid layout scheme.
>>>
>>> Section 4:
>>>
>>> "1. How do you create _equivalence classes_ (a.k.a. flows) of 
>>> traffic
>>> to which different QoS treatments are applied?"
>>>
>>> I understood that you aim for "equivalence classes" that are larger 
>>> than
>>> individual flows?
>>>
>>> Yes, modulo one’s view on what constitutes a flow in ICN terms. 
>>> This is
>>> one reason I tried to minimize the use of the term “flow” in 
>>> favor of
>>> “equivalence class” since the latter captures the actual math of 
>>> which
>>> packets are treated the same way. I don’t want to fall into the 
>>> ancient
>>> argument from TCP/IP where you could, if you were crazy enough, take
>>> packets of the same TCP “flow” and put them in different 
>>> diffserv classes
>>> and hence give them different QoS treatments.
>>>
>>> ... see Section 6.1:
>>>
>>> "First and foremost, hierarchical names are a much richer basis for
>>> specifying equivalence classes than IP 5-tuples. The IP address (or
>>> prefix) can only separate traffic by topology to the granularity of
>>> hosts, and not express actual computational instances nor sets of
>>> data."
>>>
>>> -> strong agreement from my side ...
>>>
>>> Great. From this I assume you don’t see the need for any change 
>>> here?
>>>
>>> Section 6.3:
>>>
>>> "Such a QoS treatment for ICN could invoke
>>> native ICN mechanisms, none of which are present in IP, such as:
>>> ..."
>>>
>>> I would suggest to add the option of "correlating cache utilization 
>>> with
>>> forwarding resources" to the enumeration.
>>>
>>> Good idea, will do. I think “coordinating” might be better in 
>>> this
>>> context though than “correlating”.
>>>
>>> Section 6.5:
>>>
>>> "For example, cached content for a given equivalence class
>>> can be considered _fate shared_ in a cache whereby objects from the
>>> same equivalence class are purged as a group rather than
>>> individually."
>>>
>>> This seems to indicate that all content items of a given equivalence
>>> class are governed by the same timing (belong to the same packet 
>>> train),
>>> which need not be the case.
>>>
>>> Anil complained a bit about this as well. See my response for some 
>>> more
>>> context, especially about some of the peculiarities of NAND flash as 
>>> a
>>> cache memory technology. My intention is to point out that explicit
>>> equivalence classes allow you to fate share cache space, not that 
>>> you MUST
>>> do it, or that the fate sharing is the primary discriminator in 
>>> driving an
>>> eviction algorithm. Other things are more important, such as RCT, 
>>> which
>>> will capture, at least loosely, the sliding horizon of packet 
>>> trains. My
>>> thought was that one’s basic algorithm would pick a Data packet to 
>>> evict,
>>> and then if you were under pressure to free more space, evict the
>>> fate-shared objects as well.
>>>
>>> I don’t want to get into a long digression into cache replacement
>>> algorithms. How about if I just add the caveat that you do this only 
>>> when
>>> your cache is under pressure and you are endanger of thrashing?
>>>
>>> So “…This can recover cache space more quickly and at lower 
>>> overhead than
>>> pure per-object replacement when a cache is under extreme pressure 
>>> and in
>>> danger of thrashing.“
>>>
>>> "In addition, since the
>>> forwarder remembers the QoS treatment for each pending Interest in
>>> its PIT, the above cache controls can be augmented by policy to
>>> prefer retention of cached content for some equivalence classes as
>>> part of the cache replacement algorithm."
>>>
>>> I find this a bit hard to parse: Are you referring to additional 
>>> cache
>>> policy instructions in PIT state or to correlating PIT treatment 
>>> with cache
>>> replacement?
>>>
>>> Neither, actually. I’m saying that the QoS treatment kept in the 
>>> PIT that
>>> is used to drive resources on the downstream link can be used to 
>>> affect the
>>> caching decision as well. You don’t need to only pay attention to 
>>> what the
>>> producer put in his RCT. Any suggestions for how to say this better?
>>>
>>> Section 7: Discussion of consumer/producer/operator roles
>>>
>>> In the context of a larger (multi-AS) network, I must confess to be 
>>> a bit
>>> lost here ... or more specifically: hesitant.
>>>
>>> I can follow with consumers keeping control over their access links 
>>> and
>>> maybe steering edge caching. It may also be tempting to enable 
>>> producers to
>>> characterize the nature of their content (even though this needs 
>>> some
>>> business model to prevent producers from just declaring every piece 
>>> as
>>> "highest priority"). I do see network operators in the role of 
>>> optimizing
>>> traffic dissemination/service w.r.t. their customers - the latter 
>>> getting
>>> complex, though, when looking at transit providers.
>>>
>>> This is why I think we can avoid the trap of simple remarking just
>>> because that’s all IP could do. One could have a stack, or simply 
>>> preserve
>>> the original QoS treatment while allowing any ISP, including transit 
>>> ISPs,
>>> to re-classify (e.g. through EC3 style aggregation) and/or provide 
>>> AS-local
>>> QoS treatments.
>>>
>>> The current Internet situation is afaik that large content providers 
>>> make
>>> dedicated peering agreements with large edge providers to shuffle 
>>> their
>>> content to the consumers at improved service quality. Such 
>>> operations could
>>> translate to, e.g., cache reservations, but would be subject to 
>>> contracts
>>> between operators.
>>>
>>> Sure. If you want full-blown CDNI it’s unlikely all that state 
>>> would be
>>> carried in the layer3 data plane packets anyway.
>>>
>>> The maybe obvious interest to democratize content distribution 
>>> services
>>> on a more fine-granular, individual level (i.e., not as business 
>>> contracts
>>> between giants) seems like a huge remaining challenge.
>>>
>>> Sure. Based on the above, I’ll take these as important musings as 
>>> opposed
>>> to something that needs to be explicitly added to a QoS architecture
>>> document.
>>>
>>> I find this is extremely interesting, though, and we should aim for 
>>> more
>>> reflection and ideas in the near future!
>>>
>>> See my upcoming email on how I think ICNRG might want to deal with 
>>> this
>>> piece of work of mine.
>>>
>>> Best,
>>> Thomas
>>>
>>> Thanks again for really insightful comments!
>>> DaveO.
>>>
>>> On 17/11/2019 09:52, Anil Jangam (anjangam) wrote:
>>>
>>> Hello Dave,
>>>
>>> We have reviewed the QoS arch draft and added our comments in the
>>> attached word document.
>>>
>>> Thank you,
>>> /anil.
>>>
>>>  This has a number of edits based on comments received so far 
>>> (not
>>> enough!) and offline discussions with a few people.
>>> The major addition is a discussion of what the implications of
>>> incorporating INTserv-like QoS features into CCNx or NDN might be.
>>> Comments please!!!!!
>>> DaveO
>>> Forwarded message:
>>>> From: [e-mail]
>>>> To: [e-mail]
>>>> Subject: I-D Action: draft-oran-icnrg-qosarch-02.txt
>>>> Date: Sat, 12 Oct 2019 08:31:59 -0700
>>>>
>>>> A New Internet-Draft is available from the on-line Internet-Drafts
>>>> directories.
>>>>
>>>>
>>>> Title : Considerations in the development of a QoS
>>>> Architecture for CCNx-like ICN protocols
>>>> Author : Dave Oran
>>>> Filename : draft-oran-icnrg-qosarch-02.txt
>>>> Pages : 24
>>>> Date : 2019-10-12
>>>>
>>>> Abstract:
>>>> This is a position paper. It documents the author's personal views
>>>> on how Quality of Service (QoS) capabilities ought to be
>>>> accommodated
>>>> in ICN protocols like CCNx or NDN which employ flow-balanced
>>>> Interest/Data exchanges and hop-by-hop forwarding state as their
>>>> fundamental machinery. It argues that such protocols demand a
>>>> substantially different approach to QoS from that taken in TCP/IP,
>>>> and proposes specific design patterns to achieve both
>>>> classification
>>>> and differentiated QoS treatment on both a flow and aggregate
>>>> basis.
>>>> It also considers the effect of caches as a resource in addition to
>>>> memory, CPU and link bandwidth that should be subject to explicitly
>>>> unfair resource allocation. The proposed methods are intended to
>>>> operate purely at the network layer, providing the primitives
>>>> needed
>>>> to achieve both transport and higher layer QoS objectives. It
>>>> explicitly excludes any discussion of Quality of Experience (QoE)
>>>> which can only be assessed and controlled at the application layer
>>>> or
>>>> above.
>>>>
>>>>
>>>> The IETF datatracker status page for this draft is:
>>>> https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/
>>>>
>>>> There are also htmlized versions available at:
>>>> https://tools.ietf.org/html/draft-oran-icnrg-qosarch-02
>>>> https://datatracker.ietf.org/doc/html/draft-oran-icnrg-qosarch-02
>>>>
>>>> A diff from the previous version is available at:
>>>> https://www.ietf.org/rfcdiff?url2=draft-oran-icnrg-qosarch-02
>>>>
>>>>
>>>> Please note that it may take a couple of minutes from the time of
>>>> submission
>>>> until the htmlized version and diff are available at 
>>>> tools.ietf.org.
>>>>
>>>> Internet-Drafts are also available by anonymous FTP at:
>>>> ftp://ftp.ietf.org/internet-drafts/
>>>>
>>>> _______________________________________________
>>>> I-D-Announce mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/i-d-announce
>>>> Internet-Draft directories: http://www.ietf.org/shadow.html
>>>> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>>> _______________________________________________
>>> icnrg mailing list
>>> [e-mail]
>>> https://www.irtf.org/mailman/listinfo/icnrg
>>>
>>> --
>>>
>>> Prof. Dr. Thomas C. Schmidt
>>> ° Hamburg University of Applied Sciences Berliner Tor 7 °
>>> ° Dept. Informatik, Internet Technologies Group 20099 Hamburg, 
>>> Germany °
>>> ° http://inet.haw-hamburg.de/members/schmidt Fon: [phone] 
>>> °
>>>
>>> _______________________________________________
>>> icnrg mailing list
>>> [e-mail]
>>> https://www.irtf.org/mailman/listinfo/icnrg
>>>
>>> DaveO
>>> _______________________________________________
>>> icnrg mailing list
>>> [e-mail]
>>> https://www.irtf.org/mailman/listinfo/icnrg
>>>
>> DaveO
>>


> _______________________________________________
> icnrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/icnrg

DaveO