Re: [icnrg] I just submitted an update to my ICN QoSArch draft

["David R. Oran" <[e-mail]>]

On 17 Nov 2019, at 19:27, Thomas C. Schmidt wrote:

> Hi Dave,
>
> as promised, here comes some feedback on the QoSArch draft.
>
Thanks a ton!

> Typos have already been spotted by Anil, so just a general remark on 
> capitalization: C/consumers and P/producers, also I/interest etc. are 
> not always treated uniformly.
>
See my replies to Anil, - I think I fixed all the typos he pointed out.
For the consistent capitalization, I chose to have consumer and producer 
consistently lower case, and Interest capitalized. -03 will have these 
and the other changes below based on your comments.

> The following is in chronological order:
>
> Section 3.1:
>   "There is per-Interest/Data state at every hop of
>    the path and therefore for each outstanding Interest, bandwidth for
>    data returning on the inverse path can be allocated."
>
> IMO this sounds simpler than it is, even if the size of data chunks is 
> known. Traffic is returning with an unknown delay. More specifically, 
> traffic is governed by a stochastic rate equation, and stochastic 
> calculus tells us that delay and delay variation are first order 
> terms. At least in the current Internet, the delay variation attains 
> the same order of magnitude as the delay itself, thus making actual 
> traffic return rates hard to predict.
>
Well, I am certainly oversimplifying the situation. I was hoping that by 
just saying “allocated” I could avoid people like you paying 
attention and pointing out that the stochastic nature of RTTs makes 
accurate allocation difficult. For open-loop sources like IP the delay 
and delay variation are in fact of similar magnitude, which of course 
means deterministic pre-allocation will get things wrong nearly all the 
time. The situation is somewhat better for request-response closed-loop 
protocols like ICN, but your basic point still holds.

I suspect there will be other readers with congestion control clue who 
will pick up on this, so I definitely ought to add something to the 
document. I struggled with just how much, and came up with the 
following, which I will add to the end of section 3.1:

“Given the stochastic nature of round trip times, and the ubiquity of 
wireless links and encapsulation tunnels with variable bandwidth, a 
simple scheme that admits interests only based on a time-invariant 
estimate of the returning link bandwidth will perform poorly. However, 
two characteristics of NDN and CCNx-like protocols can substantially 
help to improve the accuracy and responsiveness of the bandwidth 
allocation:

1. RTT is bounded by the Interest lifetime, which puts an upper bound on 
the RTT uncertainty for any given Interest/Data exchange. If Interest 
lifetimes are kept reasonably short (a few RTTs) the allocations do not 
have to deal with an arbitrarily long tail. One could in fact do a 
deterministic allocation on this basis, but the result would be highly 
pessimistic. Nevertheless, having a cut-off does improve the performance 
of an optimistic allocation scheme.

2. Returning Data packets can be congestion marked by an ECN-like 
marking scheme if the inverse link starts experiencing long queue 
occupancy or other congestion indication. Unlike TCP/IP, where the rate 
adjustment can only be done end-to-end, this feedback is usable 
immediately by the downstream ICN forwarder and the Interest shaping 
rate lowered after a single link RTT. This may allow less pessimistic 
rate adjustment schemes than the AIMD with .5 multiplier that is used on 
TCP/IP networks. It also allows the rate adjustments to be spread more 
accurately among the Interest/Data flows traversing a link sending 
congestion signals.”

What do you think? This will go in -03 unless you have further comments 
soon.


> Section 4, Table 2: I guess this still needs some love ;)
>
xml2rfc gives me little to no control over text table formatting. I’m 
going to leave this for now since I’m not motivated to do a lot of 
work to defeat this stupid layout scheme.

> Section 4:
>
>  "1.  How do you create _equivalence classes_ (a.k.a. flows) of 
> traffic
>       to which different QoS treatments are applied?"
>
> I understood that you aim for "equivalence classes" that are larger 
> than individual flows?
Yes, modulo one’s view on what constitutes a flow in ICN terms. This 
is one reason I tried to minimize the use of the term “flow” in 
favor of “equivalence class” since the latter captures the actual 
math of which packets are treated the same way. I don’t want to fall 
into the ancient argument from TCP/IP where you could, if you were crazy 
enough, take packets of the same TCP “flow” and put them in 
different diffserv classes and hence give them different QoS treatments.


> ... see Section 6.1:
>
>  "First and foremost, hierarchical names are a much richer basis for
>    specifying equivalence classes than IP 5-tuples.  The IP address 
> (or
>    prefix) can only separate traffic by topology to the granularity of
>    hosts, and not express actual computational instances nor sets of
>    data."
>
> -> strong agreement from my side ...
>
Great. From this I assume you don’t see the need for any change here?

> Section 6.3:
>
>  "Such a QoS treatment for ICN could invoke
>    native ICN mechanisms, none of which are present in IP, such as:
>    ..."
>
> I would suggest to add the option of "correlating cache utilization 
> with forwarding resources" to the enumeration.
>
Good idea, will do. I think “coordinating” might be better in this 
context though than “correlating”.


> Section 6.5:
>
>   "For example, cached content for a given equivalence class
>    can be considered _fate shared_ in a cache whereby objects from the
>    same equivalence class are purged as a group rather than
>    individually."
>
> This seems to indicate that all content items of a given equivalence 
> class are governed by the same timing (belong to the same packet 
> train), which need not be the case.
>
Anil complained a bit about this as well. See my response for some more 
context, especially about some of the peculiarities of NAND flash as a 
cache memory technology. My intention is to point out that explicit 
equivalence classes allow you to fate share cache space, not that you 
MUST do it, or that the fate sharing is the primary discriminator in 
driving an eviction algorithm. Other things are more important, such as 
RCT, which will capture, at least loosely, the sliding horizon of packet 
trains. My thought was that one’s basic algorithm would pick a Data 
packet to evict, and then if you were under pressure to free more space, 
evict the fate-shared objects as well.

I don’t want to get into a long digression into cache replacement 
algorithms. How about if I just add the caveat that you do this only 
when your cache is under pressure and you are endanger of thrashing?

So “…This can recover cache space more quickly and at lower overhead 
than pure per-object replacement when a cache is under extreme pressure 
and in danger of thrashing.“

>   "In addition, since the
>    forwarder remembers the QoS treatment for each pending Interest in
>    its PIT, the above cache controls can be augmented by policy to
>    prefer retention of cached content for some equivalence classes as
>    part of the cache replacement algorithm."
>
> I find this a bit hard to parse: Are you referring to additional cache 
> policy instructions in PIT state or to correlating PIT treatment with 
> cache replacement?
>
Neither, actually. I’m saying that the QoS treatment kept in the PIT 
that is used to drive resources on the downstream link can be used to 
affect the caching decision as well. You don’t need to only pay 
attention to what the producer put in his RCT. Any suggestions for how 
to say this better?


> Section 7: Discussion of consumer/producer/operator roles
>
> In the context of a larger (multi-AS) network, I must confess to be a 
> bit lost here ... or more specifically: hesitant.
>
> I can follow with consumers keeping control over their access links 
> and maybe steering edge caching. It may also be tempting to enable 
> producers to characterize the nature of their content (even though 
> this needs some business model to prevent producers from just 
> declaring every piece as "highest priority"). I do see network 
> operators in the role of optimizing traffic dissemination/service 
> w.r.t. their customers - the latter getting complex, though, when 
> looking at transit providers.
>
This is why I think we can avoid the trap of simple remarking just 
because that’s all IP could do. One could have a stack, or simply 
preserve the original QoS treatment while allowing any ISP, including 
transit ISPs, to re-classify (e.g. through EC3 style aggregation) and/or 
provide AS-local QoS treatments.

> The current Internet situation is afaik that large content providers 
> make dedicated peering agreements with large edge providers to shuffle 
> their content to the consumers at improved service quality. Such 
> operations could translate to, e.g., cache reservations, but would be 
> subject to contracts between operators.
>
Sure. If you want full-blown CDNI it’s unlikely all that state would 
be carried in the layer3 data plane packets anyway.

> The maybe obvious interest to democratize content distribution 
> services on a more fine-granular, individual level (i.e., not as 
> business contracts between giants) seems like a huge remaining 
> challenge.
>
Sure. Based on the above, I’ll take these as important musings as 
opposed to something that needs to be explicitly added to a QoS 
architecture document.

> I find this is extremely interesting, though, and we should aim for 
> more reflection and ideas in the near future!
>
See my upcoming email on how I think ICNRG might want to deal with this 
piece of work of mine.

> Best,
>  Thomas
>
Thanks again for really insightful comments!
DaveO.

>
> On 17/11/2019 09:52, Anil Jangam (anjangam) wrote:
>> Hello Dave,
>>
>> We have reviewed the QoS arch draft and added our comments in the 
>> attached word document.
>>
>> Thank you,
>> /anil.
>>
>>     This has a number of edits based on comments received so far 
>> (not
>>      enough!) and offline discussions with a few people.
>>          The major addition is a discussion of what the implications 
>> of
>>      incorporating INTserv-like QoS features into CCNx or NDN might 
>> be.
>>          Comments please!!!!!
>>          DaveO
>>          Forwarded message:
>>          > From: [e-mail]
>>      > To: [e-mail]
>>      > Subject: I-D Action: draft-oran-icnrg-qosarch-02.txt
>>      > Date: Sat, 12 Oct 2019 08:31:59 -0700
>>      >
>>      > A New Internet-Draft is available from the on-line 
>> Internet-Drafts
>>      > directories.
>>      >
>>      >
>>      >         Title           : Considerations in the development of 
>> a QoS
>>      > Architecture for CCNx-like ICN protocols
>>      >         Author          : Dave Oran
>>      > 	Filename        : draft-oran-icnrg-qosarch-02.txt
>>      > 	Pages           : 24
>>      > 	Date            : 2019-10-12
>>      >
>>      > Abstract:
>>      >    This is a position paper.  It documents the author's 
>> personal views
>>      >    on how Quality of Service (QoS) capabilities ought to be
>>      > accommodated
>>      >    in ICN protocols like CCNx or NDN which employ 
>> flow-balanced
>>      >    Interest/Data exchanges and hop-by-hop forwarding state as 
>> their
>>      >    fundamental machinery.  It argues that such protocols 
>> demand a
>>      >    substantially different approach to QoS from that taken in 
>> TCP/IP,
>>      >    and proposes specific design patterns to achieve both
>>      > classification
>>      >    and differentiated QoS treatment on both a flow and 
>> aggregate
>>      > basis.
>>      >    It also considers the effect of caches as a resource in 
>> addition to
>>      >    memory, CPU and link bandwidth that should be subject to 
>> explicitly
>>      >    unfair resource allocation.  The proposed methods are 
>> intended to
>>      >    operate purely at the network layer, providing the 
>> primitives
>>      > needed
>>      >    to achieve both transport and higher layer QoS objectives.  
>> It
>>      >    explicitly excludes any discussion of Quality of Experience 
>> (QoE)
>>      >    which can only be assessed and controlled at the 
>> application layer
>>      > or
>>      >    above.
>>      >
>>      >
>>      > The IETF datatracker status page for this draft is:
>>      > https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/
>>      >
>>      > There are also htmlized versions available at:
>>      > https://tools.ietf.org/html/draft-oran-icnrg-qosarch-02
>>      > 
>> https://datatracker.ietf.org/doc/html/draft-oran-icnrg-qosarch-02
>>      >
>>      > A diff from the previous version is available at:
>>      > https://www.ietf.org/rfcdiff?url2=draft-oran-icnrg-qosarch-02
>>      >
>>      >
>>      > Please note that it may take a couple of minutes from the time 
>> of
>>      > submission
>>      > until the htmlized version and diff are available at 
>> tools.ietf.org.
>>      >
>>      > Internet-Drafts are also available by anonymous FTP at:
>>      > ftp://ftp.ietf.org/internet-drafts/
>>      >
>>      > _______________________________________________
>>      > I-D-Announce mailing list
>>      > [e-mail]
>>      > https://www.ietf.org/mailman/listinfo/i-d-announce
>>      > Internet-Draft directories: http://www.ietf.org/shadow.html
>>      > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>>          _______________________________________________
>>      icnrg mailing list
>>      [e-mail]
>>      https://www.irtf.org/mailman/listinfo/icnrg
>>
>
> -- 
>
> Prof. Dr. Thomas C. Schmidt
> ° Hamburg University of Applied Sciences                  Berliner 
> Tor 7 °
> ° Dept. Informatik, Internet Technologies Group   20099 Hamburg, 
> Germany °
> ° http://inet.haw-hamburg.de/members/schmidt      Fon: 
> [phone] °
>
> _______________________________________________
> icnrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/icnrg

DaveO