Re: [icnrg] I just submitted an update to my ICN QoSArch draft

["David R. Oran" <[e-mail]>]

Thanks Luca, very helpful.
Couple of followup questions:

1. Despite the caveat that I’m obviously oversimplifying, you point 
out that I’m not being “100% accurate” in my new text about 
bandwidth allocation. Could you perhaps suggest some (hopefully 
terse/simple) wording that would improve the accuracy?

2. The background giving theory justification for the assertions about 
ICN congestion control advantages might be really useful for some 
readers. Might you suggest one or two references I could add?

Best,
DaveO.

On 29 Nov 2019, at 10:43, Luca Muscariello wrote:

> On Thu, Nov 28, 2019 at 5:12 PM David R. Oran <[e-mail]> 
> wrote:
>
>> On 17 Nov 2019, at 19:27, Thomas C. Schmidt wrote:
>>
>> Hi Dave,
>>
>> as promised, here comes some feedback on the QoSArch draft.
>>
>> Thanks a ton!
>>
>> Typos have already been spotted by Anil, so just a general remark on
>> capitalization: C/consumers and P/producers, also I/interest etc. are 
>> not
>> always treated uniformly.
>>
>> See my replies to Anil, - I think I fixed all the typos he pointed 
>> out.
>> For the consistent capitalization, I chose to have consumer and 
>> producer
>> consistently lower case, and Interest capitalized. -03 will have 
>> these and
>> the other changes below based on your comments.
>>
>> The following is in chronological order:
>>
>> Section 3.1:
>> "There is per-Interest/Data state at every hop of
>> the path and therefore for each outstanding Interest, bandwidth for
>> data returning on the inverse path can be allocated."
>>
>> IMO this sounds simpler than it is, even if the size of data chunks 
>> is
>> known. Traffic is returning with an unknown delay. More specifically,
>> traffic is governed by a stochastic rate equation, and stochastic 
>> calculus
>> tells us that delay and delay variation are first order terms. At 
>> least in
>> the current Internet, the delay variation attains the same order of
>> magnitude as the delay itself, thus making actual traffic return 
>> rates hard
>> to predict.
>>
>> Well, I am certainly oversimplifying the situation. I was hoping that 
>> by
>> just saying “allocated” I could avoid people like you paying 
>> attention and
>> pointing out that the stochastic nature of RTTs makes accurate 
>> allocation
>> difficult. For open-loop sources like IP the delay and delay 
>> variation are
>> in fact of similar magnitude, which of course means deterministic
>> pre-allocation will get things wrong nearly all the time. The 
>> situation is
>> somewhat better for request-response closed-loop protocols like ICN, 
>> but
>> your basic point still holds.
>>
>> I suspect there will be other readers with congestion control clue 
>> who
>> will pick up on this, so I definitely ought to add something to the
>> document. I struggled with just how much, and came up with the 
>> following,
>> which I will add to the end of section 3.1:
>>
>> “Given the stochastic nature of round trip times, and the ubiquity 
>> of
>> wireless links and encapsulation tunnels with variable bandwidth, a 
>> simple
>> scheme that admits interests only based on a time-invariant estimate 
>> of the
>> returning link bandwidth will perform poorly. However, two 
>> characteristics
>> of NDN and CCNx-like protocols can substantially help to improve the
>> accuracy and responsiveness of the bandwidth allocation:
>>
>>    1.
>>
>>    RTT is bounded by the Interest lifetime, which puts an upper bound 
>> on
>>    the RTT uncertainty for any given Interest/Data exchange. If 
>> Interest
>>    lifetimes are kept reasonably short (a few RTTs) the allocations 
>> do not
>>    have to deal with an arbitrarily long tail. One could in fact do a
>>    deterministic allocation on this basis, but the result would be 
>> highly
>>    pessimistic. Nevertheless, having a cut-off does improve the 
>> performance of
>>    an optimistic allocation scheme.
>>    2.
>>
>>    Returning Data packets can be congestion marked by an ECN-like 
>> marking
>>    scheme if the inverse link starts experiencing long queue 
>> occupancy or
>>    other congestion indication. Unlike TCP/IP, where the rate 
>> adjustment can
>>    only be done end-to-end, this feedback is usable immediately by 
>> the
>>    downstream ICN forwarder and the Interest shaping rate lowered 
>> after a
>>    single link RTT. This may allow less pessimistic rate adjustment 
>> schemes
>>    than the AIMD with .5 multiplier that is used on TCP/IP networks. 
>> It also
>>    allows the rate adjustments to be spread more accurately among the
>>    Interest/Data flows traversing a link sending congestion 
>> signals.”
>>
>>
> The instantaneous number of pending interests (N_t) is the integral of 
> the
> difference between interest(I_t) and data rates (D_t):
> N_t = \Int_[0,t] max(0, I_u - D_t)du.
>
> This is a virtual queue measured at a given node.
> As these rates can be associated naturally on a per-packet (1:1) basis 
> and
> aggregated and disaggregated as needed (per interface, per prefix etc)
> several virtual queues can be computed. These are the equivalence 
> classes
> according to Dave's definition.
>
> These virtual queues can be associated naturally to the Lagrange
> multipliers associated to the congested links
> in the network and therefore they are equivalent to ECN signals, 
> which, in
> the case of TCP, can also be associated
> to congested links as they are fundamentally associated to the 
> capacity
> constraints in a global network optimization problem.
>
> There is previous work where these concepts have been formally 
> defined,
> that I can report if needed.
>
> Even if delays follow a stochastic process that does not mean they 
> cannot
> be used in practice.
> Stochastic is not a synonym of unpredictable. In the case of a virtual
> queues (as defined above),
> the integral formulation is a lot more robust than delay measurements
> (first or second order statistics)
> and could be used in theory to optimize resource allocation.
>
> If Dave's text may be not 100% accurate, I think the idea is valid and
> would be good to have it
> spelled out entirely in the document, with maybe reference with the
> theoretical concepts.
> This is the fundamental underlying advantage, with a very solid 
> theoretical
> explanation
> of local flow balance and why it is formally superior to IP to control
> congestion.
>
> Nonetheless, I agree with Thomas when he says, "this sounds simpler 
> than it
> is"!
>
>
>
>
>>
>>
>> What do you think? This will go in -03 unless you have further 
>> comments
>> soon.
>>
>> Section 4, Table 2: I guess this still needs some love ;)
>>
>> xml2rfc gives me little to no control over text table formatting. 
>> I’m
>> going to leave this for now since I’m not motivated to do a lot of 
>> work to
>> defeat this stupid layout scheme.
>>
>> Section 4:
>>
>> "1. How do you create _equivalence classes_ (a.k.a. flows) of traffic
>> to which different QoS treatments are applied?"
>>
>> I understood that you aim for "equivalence classes" that are larger 
>> than
>> individual flows?
>>
>> Yes, modulo one’s view on what constitutes a flow in ICN terms. 
>> This is
>> one reason I tried to minimize the use of the term “flow” in 
>> favor of
>> “equivalence class” since the latter captures the actual math of 
>> which
>> packets are treated the same way. I don’t want to fall into the 
>> ancient
>> argument from TCP/IP where you could, if you were crazy enough, take
>> packets of the same TCP “flow” and put them in different diffserv 
>> classes
>> and hence give them different QoS treatments.
>>
>> ... see Section 6.1:
>>
>> "First and foremost, hierarchical names are a much richer basis for
>> specifying equivalence classes than IP 5-tuples. The IP address (or
>> prefix) can only separate traffic by topology to the granularity of
>> hosts, and not express actual computational instances nor sets of
>> data."
>>
>> -> strong agreement from my side ...
>>
>> Great. From this I assume you don’t see the need for any change 
>> here?
>>
>> Section 6.3:
>>
>> "Such a QoS treatment for ICN could invoke
>> native ICN mechanisms, none of which are present in IP, such as:
>> ..."
>>
>> I would suggest to add the option of "correlating cache utilization 
>> with
>> forwarding resources" to the enumeration.
>>
>> Good idea, will do. I think “coordinating” might be better in 
>> this context
>> though than “correlating”.
>>
>> Section 6.5:
>>
>> "For example, cached content for a given equivalence class
>> can be considered _fate shared_ in a cache whereby objects from the
>> same equivalence class are purged as a group rather than
>> individually."
>>
>> This seems to indicate that all content items of a given equivalence 
>> class
>> are governed by the same timing (belong to the same packet train), 
>> which
>> need not be the case.
>>
>> Anil complained a bit about this as well. See my response for some 
>> more
>> context, especially about some of the peculiarities of NAND flash as 
>> a
>> cache memory technology. My intention is to point out that explicit
>> equivalence classes allow you to fate share cache space, not that you 
>> MUST
>> do it, or that the fate sharing is the primary discriminator in 
>> driving an
>> eviction algorithm. Other things are more important, such as RCT, 
>> which
>> will capture, at least loosely, the sliding horizon of packet trains. 
>> My
>> thought was that one’s basic algorithm would pick a Data packet to 
>> evict,
>> and then if you were under pressure to free more space, evict the
>> fate-shared objects as well.
>>
>> I don’t want to get into a long digression into cache replacement
>> algorithms. How about if I just add the caveat that you do this only 
>> when
>> your cache is under pressure and you are endanger of thrashing?
>>
>> So “…This can recover cache space more quickly and at lower 
>> overhead than
>> pure per-object replacement when a cache is under extreme pressure 
>> and in
>> danger of thrashing.“
>>
>> "In addition, since the
>> forwarder remembers the QoS treatment for each pending Interest in
>> its PIT, the above cache controls can be augmented by policy to
>> prefer retention of cached content for some equivalence classes as
>> part of the cache replacement algorithm."
>>
>> I find this a bit hard to parse: Are you referring to additional 
>> cache
>> policy instructions in PIT state or to correlating PIT treatment with 
>> cache
>> replacement?
>>
>> Neither, actually. I’m saying that the QoS treatment kept in the 
>> PIT that
>> is used to drive resources on the downstream link can be used to 
>> affect the
>> caching decision as well. You don’t need to only pay attention to 
>> what the
>> producer put in his RCT. Any suggestions for how to say this better?
>>
>> Section 7: Discussion of consumer/producer/operator roles
>>
>> In the context of a larger (multi-AS) network, I must confess to be a 
>> bit
>> lost here ... or more specifically: hesitant.
>>
>> I can follow with consumers keeping control over their access links 
>> and
>> maybe steering edge caching. It may also be tempting to enable 
>> producers to
>> characterize the nature of their content (even though this needs some
>> business model to prevent producers from just declaring every piece 
>> as
>> "highest priority"). I do see network operators in the role of 
>> optimizing
>> traffic dissemination/service w.r.t. their customers - the latter 
>> getting
>> complex, though, when looking at transit providers.
>>
>> This is why I think we can avoid the trap of simple remarking just 
>> because
>> that’s all IP could do. One could have a stack, or simply preserve 
>> the
>> original QoS treatment while allowing any ISP, including transit 
>> ISPs, to
>> re-classify (e.g. through EC3 style aggregation) and/or provide 
>> AS-local
>> QoS treatments.
>>
>> The current Internet situation is afaik that large content providers 
>> make
>> dedicated peering agreements with large edge providers to shuffle 
>> their
>> content to the consumers at improved service quality. Such operations 
>> could
>> translate to, e.g., cache reservations, but would be subject to 
>> contracts
>> between operators.
>>
>> Sure. If you want full-blown CDNI it’s unlikely all that state 
>> would be
>> carried in the layer3 data plane packets anyway.
>>
>> The maybe obvious interest to democratize content distribution 
>> services on
>> a more fine-granular, individual level (i.e., not as business 
>> contracts
>> between giants) seems like a huge remaining challenge.
>>
>> Sure. Based on the above, I’ll take these as important musings as 
>> opposed
>> to something that needs to be explicitly added to a QoS architecture
>> document.
>>
>> I find this is extremely interesting, though, and we should aim for 
>> more
>> reflection and ideas in the near future!
>>
>> See my upcoming email on how I think ICNRG might want to deal with 
>> this
>> piece of work of mine.
>>
>> Best,
>> Thomas
>>
>> Thanks again for really insightful comments!
>> DaveO.
>>
>> On 17/11/2019 09:52, Anil Jangam (anjangam) wrote:
>>
>> Hello Dave,
>>
>> We have reviewed the QoS arch draft and added our comments in the 
>> attached
>> word document.
>>
>> Thank you,
>> /anil.
>>
>>  This has a number of edits based on comments received so far (not
>> enough!) and offline discussions with a few people.
>> The major addition is a discussion of what the implications of
>> incorporating INTserv-like QoS features into CCNx or NDN might be.
>> Comments please!!!!!
>> DaveO
>> Forwarded message:
>>> From: [e-mail]
>>> To: [e-mail]
>>> Subject: I-D Action: draft-oran-icnrg-qosarch-02.txt
>>> Date: Sat, 12 Oct 2019 08:31:59 -0700
>>>
>>> A New Internet-Draft is available from the on-line Internet-Drafts
>>> directories.
>>>
>>>
>>> Title : Considerations in the development of a QoS
>>> Architecture for CCNx-like ICN protocols
>>> Author : Dave Oran
>>> Filename : draft-oran-icnrg-qosarch-02.txt
>>> Pages : 24
>>> Date : 2019-10-12
>>>
>>> Abstract:
>>> This is a position paper. It documents the author's personal views
>>> on how Quality of Service (QoS) capabilities ought to be
>>> accommodated
>>> in ICN protocols like CCNx or NDN which employ flow-balanced
>>> Interest/Data exchanges and hop-by-hop forwarding state as their
>>> fundamental machinery. It argues that such protocols demand a
>>> substantially different approach to QoS from that taken in TCP/IP,
>>> and proposes specific design patterns to achieve both
>>> classification
>>> and differentiated QoS treatment on both a flow and aggregate
>>> basis.
>>> It also considers the effect of caches as a resource in addition to
>>> memory, CPU and link bandwidth that should be subject to explicitly
>>> unfair resource allocation. The proposed methods are intended to
>>> operate purely at the network layer, providing the primitives
>>> needed
>>> to achieve both transport and higher layer QoS objectives. It
>>> explicitly excludes any discussion of Quality of Experience (QoE)
>>> which can only be assessed and controlled at the application layer
>>> or
>>> above.
>>>
>>>
>>> The IETF datatracker status page for this draft is:
>>> https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/
>>>
>>> There are also htmlized versions available at:
>>> https://tools.ietf.org/html/draft-oran-icnrg-qosarch-02
>>> https://datatracker.ietf.org/doc/html/draft-oran-icnrg-qosarch-02
>>>
>>> A diff from the previous version is available at:
>>> https://www.ietf.org/rfcdiff?url2=draft-oran-icnrg-qosarch-02
>>>
>>>
>>> Please note that it may take a couple of minutes from the time of
>>> submission
>>> until the htmlized version and diff are available at tools.ietf.org.
>>>
>>> Internet-Drafts are also available by anonymous FTP at:
>>> ftp://ftp.ietf.org/internet-drafts/
>>>
>>> _______________________________________________
>>> I-D-Announce mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/i-d-announce
>>> Internet-Draft directories: http://www.ietf.org/shadow.html
>>> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>
>> --
>>
>> Prof. Dr. Thomas C. Schmidt
>> ° Hamburg University of Applied Sciences Berliner Tor 7 °
>> ° Dept. Informatik, Internet Technologies Group 20099 Hamburg, 
>> Germany °
>> ° http://inet.haw-hamburg.de/members/schmidt Fon: [phone] 
>> °
>>
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>
>> DaveO
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>



DaveO