Re: [icnrg] I just submitted an update to my ICN QoSArch draft

[Luca Muscariello <[e-mail]>]

On Sat, Nov 30, 2019 at 2:53 PM David R. Oran <[e-mail]> wrote:

> Thanks Luca, very helpful.
> Couple of followup questions:
>
>    1.
>
>    Despite the caveat that I’m obviously oversimplifying, you point out
>    that I’m not being “100% accurate” in my new text about bandwidth
>    allocation. Could you perhaps suggest some (hopefully terse/simple) wording
>    that would improve the accuracy?
>
>
As short as possible

"There is per-Interest/Data state at every hop of the path and therefore
outstanding Interests
provide information that can be used to optimize resource allocation for
data returning on the inverse path, such as
bandwidth sharing, prioritization and overload control".


>
>    1.
>
>    The background giving theory justification for the assertions about
>    ICN congestion control advantages might be really useful for some readers.
>    Might you suggest one or two references I could add?
>
>
This is one example about how and why outstanding interests are related to
network resources optimization.
Full disclosure, I'm a co-author of this paper.

https://doi.org/10.1016/j.comnet.2016.09.012


Luca



>
> Best,
> DaveO.
>
> On 29 Nov 2019, at 10:43, Luca Muscariello wrote:
>
>
>
> On Thu, Nov 28, 2019 at 5:12 PM David R. Oran <[e-mail]>
> wrote:
>
>> On 17 Nov 2019, at 19:27, Thomas C. Schmidt wrote:
>>
>> Hi Dave,
>>
>> as promised, here comes some feedback on the QoSArch draft.
>>
>> Thanks a ton!
>>
>> Typos have already been spotted by Anil, so just a general remark on
>> capitalization: C/consumers and P/producers, also I/interest etc. are not
>> always treated uniformly.
>>
>> See my replies to Anil, - I think I fixed all the typos he pointed out.
>> For the consistent capitalization, I chose to have consumer and producer
>> consistently lower case, and Interest capitalized. -03 will have these and
>> the other changes below based on your comments.
>>
>> The following is in chronological order:
>>
>> Section 3.1:
>> "There is per-Interest/Data state at every hop of
>> the path and therefore for each outstanding Interest, bandwidth for
>> data returning on the inverse path can be allocated."
>>
>> IMO this sounds simpler than it is, even if the size of data chunks is
>> known. Traffic is returning with an unknown delay. More specifically,
>> traffic is governed by a stochastic rate equation, and stochastic calculus
>> tells us that delay and delay variation are first order terms. At least in
>> the current Internet, the delay variation attains the same order of
>> magnitude as the delay itself, thus making actual traffic return rates hard
>> to predict.
>>
>> Well, I am certainly oversimplifying the situation. I was hoping that by
>> just saying “allocated” I could avoid people like you paying attention and
>> pointing out that the stochastic nature of RTTs makes accurate allocation
>> difficult. For open-loop sources like IP the delay and delay variation are
>> in fact of similar magnitude, which of course means deterministic
>> pre-allocation will get things wrong nearly all the time. The situation is
>> somewhat better for request-response closed-loop protocols like ICN, but
>> your basic point still holds.
>>
>> I suspect there will be other readers with congestion control clue who
>> will pick up on this, so I definitely ought to add something to the
>> document. I struggled with just how much, and came up with the following,
>> which I will add to the end of section 3.1:
>>
>> “Given the stochastic nature of round trip times, and the ubiquity of
>> wireless links and encapsulation tunnels with variable bandwidth, a simple
>> scheme that admits interests only based on a time-invariant estimate of the
>> returning link bandwidth will perform poorly. However, two characteristics
>> of NDN and CCNx-like protocols can substantially help to improve the
>> accuracy and responsiveness of the bandwidth allocation:
>>
>>    1.
>>
>>    RTT is bounded by the Interest lifetime, which puts an upper bound on
>>    the RTT uncertainty for any given Interest/Data exchange. If Interest
>>    lifetimes are kept reasonably short (a few RTTs) the allocations do not
>>    have to deal with an arbitrarily long tail. One could in fact do a
>>    deterministic allocation on this basis, but the result would be highly
>>    pessimistic. Nevertheless, having a cut-off does improve the performance of
>>    an optimistic allocation scheme.
>>    2.
>>
>>    Returning Data packets can be congestion marked by an ECN-like
>>    marking scheme if the inverse link starts experiencing long queue occupancy
>>    or other congestion indication. Unlike TCP/IP, where the rate adjustment
>>    can only be done end-to-end, this feedback is usable immediately by the
>>    downstream ICN forwarder and the Interest shaping rate lowered after a
>>    single link RTT. This may allow less pessimistic rate adjustment schemes
>>    than the AIMD with .5 multiplier that is used on TCP/IP networks. It also
>>    allows the rate adjustments to be spread more accurately among the
>>    Interest/Data flows traversing a link sending congestion signals.”
>>
>>
> The instantaneous number of pending interests (N_t) is the integral of the
> difference between interest(I_t) and data rates (D_t):
> N_t = \Int_[0,t] max(0, I_u - D_t)du.
>
> This is a virtual queue measured at a given node.
> As these rates can be associated naturally on a per-packet (1:1) basis and
> aggregated and disaggregated as needed (per interface, per prefix etc)
> several virtual queues can be computed. These are the equivalence classes
> according to Dave's definition.
>
> These virtual queues can be associated naturally to the Lagrange
> multipliers associated to the congested links
> in the network and therefore they are equivalent to ECN signals, which, in
> the case of TCP, can also be associated
> to congested links as they are fundamentally associated to the capacity
> constraints in a global network optimization problem.
>
> There is previous work where these concepts have been formally defined,
> that I can report if needed.
>
> Even if delays follow a stochastic process that does not mean they cannot
> be used in practice.
> Stochastic is not a synonym of unpredictable. In the case of a virtual
> queues (as defined above),
> the integral formulation is a lot more robust than delay measurements
> (first or second order statistics)
> and could be used in theory to optimize resource allocation.
>
> If Dave's text may be not 100% accurate, I think the idea is valid and
> would be good to have it
> spelled out entirely in the document, with maybe reference with the
> theoretical concepts.
> This is the fundamental underlying advantage, with a very solid
> theoretical explanation
> of local flow balance and why it is formally superior to IP to control
> congestion.
>
> Nonetheless, I agree with Thomas when he says, "this sounds simpler than
> it is"!
>
>
>
>
>>
>>
>> What do you think? This will go in -03 unless you have further comments
>> soon.
>>
>> Section 4, Table 2: I guess this still needs some love ;)
>>
>> xml2rfc gives me little to no control over text table formatting. I’m
>> going to leave this for now since I’m not motivated to do a lot of work to
>> defeat this stupid layout scheme.
>>
>> Section 4:
>>
>> "1. How do you create _equivalence classes_ (a.k.a. flows) of traffic
>> to which different QoS treatments are applied?"
>>
>> I understood that you aim for "equivalence classes" that are larger than
>> individual flows?
>>
>> Yes, modulo one’s view on what constitutes a flow in ICN terms. This is
>> one reason I tried to minimize the use of the term “flow” in favor of
>> “equivalence class” since the latter captures the actual math of which
>> packets are treated the same way. I don’t want to fall into the ancient
>> argument from TCP/IP where you could, if you were crazy enough, take
>> packets of the same TCP “flow” and put them in different diffserv classes
>> and hence give them different QoS treatments.
>>
>> ... see Section 6.1:
>>
>> "First and foremost, hierarchical names are a much richer basis for
>> specifying equivalence classes than IP 5-tuples. The IP address (or
>> prefix) can only separate traffic by topology to the granularity of
>> hosts, and not express actual computational instances nor sets of
>> data."
>>
>> -> strong agreement from my side ...
>>
>> Great. From this I assume you don’t see the need for any change here?
>>
>> Section 6.3:
>>
>> "Such a QoS treatment for ICN could invoke
>> native ICN mechanisms, none of which are present in IP, such as:
>> ..."
>>
>> I would suggest to add the option of "correlating cache utilization with
>> forwarding resources" to the enumeration.
>>
>> Good idea, will do. I think “coordinating” might be better in this
>> context though than “correlating”.
>>
>> Section 6.5:
>>
>> "For example, cached content for a given equivalence class
>> can be considered _fate shared_ in a cache whereby objects from the
>> same equivalence class are purged as a group rather than
>> individually."
>>
>> This seems to indicate that all content items of a given equivalence
>> class are governed by the same timing (belong to the same packet train),
>> which need not be the case.
>>
>> Anil complained a bit about this as well. See my response for some more
>> context, especially about some of the peculiarities of NAND flash as a
>> cache memory technology. My intention is to point out that explicit
>> equivalence classes allow you to fate share cache space, not that you MUST
>> do it, or that the fate sharing is the primary discriminator in driving an
>> eviction algorithm. Other things are more important, such as RCT, which
>> will capture, at least loosely, the sliding horizon of packet trains. My
>> thought was that one’s basic algorithm would pick a Data packet to evict,
>> and then if you were under pressure to free more space, evict the
>> fate-shared objects as well.
>>
>> I don’t want to get into a long digression into cache replacement
>> algorithms. How about if I just add the caveat that you do this only when
>> your cache is under pressure and you are endanger of thrashing?
>>
>> So “…This can recover cache space more quickly and at lower overhead than
>> pure per-object replacement when a cache is under extreme pressure and in
>> danger of thrashing.“
>>
>> "In addition, since the
>> forwarder remembers the QoS treatment for each pending Interest in
>> its PIT, the above cache controls can be augmented by policy to
>> prefer retention of cached content for some equivalence classes as
>> part of the cache replacement algorithm."
>>
>> I find this a bit hard to parse: Are you referring to additional cache
>> policy instructions in PIT state or to correlating PIT treatment with cache
>> replacement?
>>
>> Neither, actually. I’m saying that the QoS treatment kept in the PIT that
>> is used to drive resources on the downstream link can be used to affect the
>> caching decision as well. You don’t need to only pay attention to what the
>> producer put in his RCT. Any suggestions for how to say this better?
>>
>> Section 7: Discussion of consumer/producer/operator roles
>>
>> In the context of a larger (multi-AS) network, I must confess to be a bit
>> lost here ... or more specifically: hesitant.
>>
>> I can follow with consumers keeping control over their access links and
>> maybe steering edge caching. It may also be tempting to enable producers to
>> characterize the nature of their content (even though this needs some
>> business model to prevent producers from just declaring every piece as
>> "highest priority"). I do see network operators in the role of optimizing
>> traffic dissemination/service w.r.t. their customers - the latter getting
>> complex, though, when looking at transit providers.
>>
>> This is why I think we can avoid the trap of simple remarking just
>> because that’s all IP could do. One could have a stack, or simply preserve
>> the original QoS treatment while allowing any ISP, including transit ISPs,
>> to re-classify (e.g. through EC3 style aggregation) and/or provide AS-local
>> QoS treatments.
>>
>> The current Internet situation is afaik that large content providers make
>> dedicated peering agreements with large edge providers to shuffle their
>> content to the consumers at improved service quality. Such operations could
>> translate to, e.g., cache reservations, but would be subject to contracts
>> between operators.
>>
>> Sure. If you want full-blown CDNI it’s unlikely all that state would be
>> carried in the layer3 data plane packets anyway.
>>
>> The maybe obvious interest to democratize content distribution services
>> on a more fine-granular, individual level (i.e., not as business contracts
>> between giants) seems like a huge remaining challenge.
>>
>> Sure. Based on the above, I’ll take these as important musings as opposed
>> to something that needs to be explicitly added to a QoS architecture
>> document.
>>
>> I find this is extremely interesting, though, and we should aim for more
>> reflection and ideas in the near future!
>>
>> See my upcoming email on how I think ICNRG might want to deal with this
>> piece of work of mine.
>>
>> Best,
>> Thomas
>>
>> Thanks again for really insightful comments!
>> DaveO.
>>
>> On 17/11/2019 09:52, Anil Jangam (anjangam) wrote:
>>
>> Hello Dave,
>>
>> We have reviewed the QoS arch draft and added our comments in the
>> attached word document.
>>
>> Thank you,
>> /anil.
>>
>>  This has a number of edits based on comments received so far (not
>> enough!) and offline discussions with a few people.
>> The major addition is a discussion of what the implications of
>> incorporating INTserv-like QoS features into CCNx or NDN might be.
>> Comments please!!!!!
>> DaveO
>> Forwarded message:
>> > From: [e-mail]
>> > To: [e-mail]
>> > Subject: I-D Action: draft-oran-icnrg-qosarch-02.txt
>> > Date: Sat, 12 Oct 2019 08:31:59 -0700
>> >
>> > A New Internet-Draft is available from the on-line Internet-Drafts
>> > directories.
>> >
>> >
>> > Title : Considerations in the development of a QoS
>> > Architecture for CCNx-like ICN protocols
>> > Author : Dave Oran
>> > Filename : draft-oran-icnrg-qosarch-02.txt
>> > Pages : 24
>> > Date : 2019-10-12
>> >
>> > Abstract:
>> > This is a position paper. It documents the author's personal views
>> > on how Quality of Service (QoS) capabilities ought to be
>> > accommodated
>> > in ICN protocols like CCNx or NDN which employ flow-balanced
>> > Interest/Data exchanges and hop-by-hop forwarding state as their
>> > fundamental machinery. It argues that such protocols demand a
>> > substantially different approach to QoS from that taken in TCP/IP,
>> > and proposes specific design patterns to achieve both
>> > classification
>> > and differentiated QoS treatment on both a flow and aggregate
>> > basis.
>> > It also considers the effect of caches as a resource in addition to
>> > memory, CPU and link bandwidth that should be subject to explicitly
>> > unfair resource allocation. The proposed methods are intended to
>> > operate purely at the network layer, providing the primitives
>> > needed
>> > to achieve both transport and higher layer QoS objectives. It
>> > explicitly excludes any discussion of Quality of Experience (QoE)
>> > which can only be assessed and controlled at the application layer
>> > or
>> > above.
>> >
>> >
>> > The IETF datatracker status page for this draft is:
>> > https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/
>> >
>> > There are also htmlized versions available at:
>> > https://tools.ietf.org/html/draft-oran-icnrg-qosarch-02
>> > https://datatracker.ietf.org/doc/html/draft-oran-icnrg-qosarch-02
>> >
>> > A diff from the previous version is available at:
>> > https://www.ietf.org/rfcdiff?url2=draft-oran-icnrg-qosarch-02
>> >
>> >
>> > Please note that it may take a couple of minutes from the time of
>> > submission
>> > until the htmlized version and diff are available at tools.ietf.org.
>> >
>> > Internet-Drafts are also available by anonymous FTP at:
>> > ftp://ftp.ietf.org/internet-drafts/
>> >
>> > _______________________________________________
>> > I-D-Announce mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/i-d-announce
>> > Internet-Draft directories: http://www.ietf.org/shadow.html
>> > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>
>> --
>>
>> Prof. Dr. Thomas C. Schmidt
>> ° Hamburg University of Applied Sciences Berliner Tor 7 °
>> ° Dept. Informatik, Internet Technologies Group 20099 Hamburg, Germany °
>> ° http://inet.haw-hamburg.de/members/schmidt Fon: [phone] °
>>
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>
>> DaveO
>> _______________________________________________
>> icnrg mailing list
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/icnrg
>>
> DaveO
>