[DLNEX] 答复: Various sources contributing to E2E latency/delay-- Discussion of reliable and deterministic latency attributes

[Zongning <[e-mail]>]

Hi, Bob,

Thanks for pointing out the text for reading. It is an excellent and useful survey indeed. ☺
While I am still reading the paper, I have quick (maybe naïve) questions as below.

1)       I see you adopted “the amount of data transferred” as one parameter for benchmarking. Would you think it is also important to consider the “characteristic of the flow”? e.g. some flow may have regular burst or even be totally unexpected.

2)       I am especially interested in DBCC (Delay Based Congestion Control). The paper argues it is difficult to deploy over the Internet due to potential starvation by loss-based flows. Is this one of the motivation of L4S that separates the queues for different (loss or latency sensitive) flows? Again, I’d think it is interesting to have L4S (or similar) capability abstracted and exposed to handle potentially unbounded flows, e.g. how many latency bounded flows could be supported in certain network node.

Anyway, this is a good paper for gap analysis and build new ideas upon.

Thanks.

-Ning

发件人: DLNEX [mailto:[e-mail]] 代表 Bob Briscoe
发送时间: 2016年10月28日 0:40
收件人: Linda Dunbar; Lou Berger; Pat Thaler
抄送: Pascal Thubert (pthubert); [e-mail]; Patrick Wetterwald (pwetterw); Alia Atlas; [e-mail]; [e-mail]
主题: Re: [DLNEX] Various sources contributing to E2E latency/delay-- Discussion of reliable and deterministic latency attributes

Linda,

Probably the most useful part of that paper (other than saving you reading over 300 references) is Chapter IX. It attempts to select some of the most interesting approaches, and to quantify how much of the overall latency (in selected scenarios) they remove. Both as text, and as a visualization of their relative merits (mean and variance) in some nice (deliberately approximate) bubble diagrams, using the colour coding of the categories Linda outlined below.



Bob
On 20/10/16 21:18, Linda Dunbar wrote:
Lou,

Change the subject to make the discussion more focused.

As of now, the DLNex is to explore what kind of characteristics can be exposed by network, or can be imposed by upper layer, to make the delay/latency of E2E services more predictable or more optimal. The final work may eventually fall into some existing IETF WGs. Here, I am just brainstorming some potential exposures:

This paper "Reducing Internet Latency: A Survey of Techniques and their Merits" (http://www.bobbriscoe.net/pubs.html#latency_survey ) listed multiple sources that contributing to Latency/delay for E2E services:

The sources of delay are classified into five main categories: structural delays, interaction between endpoints, delays along transmission paths, delays related to link capacities, and intra-end-host delays.

Structural delays arise from the structure of the network or the communication path that is used.
The paper shows that one E2E service can use DNS multiple times. Therefore, where DNS is located is a big contributor to the overall latency. This could be a potential exposure by the network.

Delays resulting from the interaction between endpoints include delays due to transport initialization and secure session initialization, as well as delays from recovering lost packets and from message-aggregation techniques
Is it another exposure? Is there secure channel used?

Delays along transmission paths captures the delays that may be encountered as data travels between a sender and a receiver.
If some segments along the transmission paths are DetNet enabled or OIF’s FlexEthernet enabled, this definitely should be exposed. Then the upper layer can control/reserve the resource as CCAMP has done for MPLS paths and optical channels.

Delays related to link capacities include both delays resulting from sharing limited capacity and delays from protocol inefficiencies that under-utilize capacity and therefore communication takes longer than necessary.
I assume this is the type of work that CCAMP is to address.

Intra-end-host delays are delays that occur internally within host endpoints. This includes delays due to buffering in the transport protocol stack and delays within the operating system.
This is the end hosts TCP/UDP layer.

Linda

-----Original Message-----
From: Lou Berger [mailto:[e-mail]]
Sent: Thursday, October 20, 2016 2:36 PM
To: Linda Dunbar <[e-mail]><mailto:[e-mail]>; Pat Thaler <[e-mail]><mailto:[e-mail]>
Cc: Pascal Thubert (pthubert) <[e-mail]><mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; Patrick Wetterwald (pwetterw) <[e-mail]><mailto:[e-mail]>; Alia Atlas <[e-mail]><mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Detnet] FW: New Non-WG Mailing List: DLNEX -- Discussion of reliable and deterministic latency attributes

Hi Linda,

    Could you (or anyone on DLNEX) elaborate on the type of documents you're thinking of delivering?

In detnet, we've so far been more focused on applications/use cases and data plane options.  But we're really looking forward to contributions/drafts on the information that is needed to support the control of deterministic flows.  The charter states this as:

    Data flow information model: This work will identify the
    information needed for flow establishment and control and be
    used by reservation protocols and YANG data models. The work
    will be independent from the protocol(s) used to control the
    flows (e.g. YANG+NETCONF/RESTCONF, PCEP or GMPLS).

Any drafts/contributions on this topic or willingness to help define these would be great.

If there are sufficiently developed ideas, we could probably even find time to discuss in our Seoul session.  (Or perhaps just informally, if there aren't.)

Thanks,
Lou

On 10/19/2016 7:09 PM, Linda Dunbar wrote:
>
> Pat,
>
>
>
> What you said is so true: “The network can't provide a bounded or
> deterministic latency to an unbounded flow”.
>
>
>
> DETNET charter starts with
>
> The Deterministic Networking (DetNet) Working Group focuses on
> deterministic data paths that operate over Layer 2 bridged and Layer 3
> routed segments, where such paths can provide bounds on latency, loss,
> and packet delay variation (jitter), and high reliability
>
>
>
> If I understand it correctly, DETNET focuses on the enabling
> technologies to ensure deterministic latency path across a set of nodes.
>
>
>
> E2E low latency services has many issues, such as:
>
> ·        There could be “bounded flows” that need low latency services
> mixed with other traffic in the network.
>
> ·        E2E services may have to go through many hops, links with
> some links not supporting DETNET.
>
> ·        there could be various protection/re-route mechanisms which
> all increase latency.
>
> ·        etc
>
>
>
> Some problems specific to WAN include how to isolate the various
> service traffic, and provide maximum manageability to upper layers.
>
>
>
> One company in NANOG asked why not use ICMP to mark the delay along
> the path to give upper link, ..
>
>
>
> Linda
>
>
>
>
>
> Linda
>
>
>
> *From:*Pat Thaler [mailto:[e-mail]]
> *Sent:* Wednesday, October 19, 2016 5:25 PM
> *To:* Lou Berger <[e-mail]<mailto:[e-mail]>>
> *Cc:* Alia Atlas <[e-mail]<mailto:[e-mail]>>; Patrick Wetterwald (pwetterw)
> <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; Pascal Thubert
> (pthubert) <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>;
> Linda Dunbar <[e-mail]<mailto:[e-mail]>>
> *Subject:* Re: [Detnet] FW: New Non-WG Mailing List: DLNEX --
> Discussion of reliable and deterministic latency attributes
>
>
>
> We have been having recent discussions about latency such as what type
> of latency is important.
>
>
>
> As far as the "feasibility exercise for achieving reliable and
> deterministic latency through a network element or a segment" that
> overlaps very much what DetNet and IEEE 802.1 TSN have been doing. I
> believe we have already shown that it is feasible and are not on
> specifying how to do that. I'm concerned about the overlap between
> what that paragraph describes and our work.
>
>
>
> There could be useful work on communication from the upper layer of
> the flow characteristics, behavior and requirements. There probably is
> some gap to fill that would help our work. I'm concerned that the list
> description focuses only on exposing lower layer characteristics to
> the upper layer. The network can't provide a bounded or deterministic
> latency to an unbounded flow - to determine what latency can be
> provided, one has to know something about the bandwidth utilization of
> the flow.
>
>
>
> Regards,
>
> Pat
>
>
>
> On Wed, Oct 19, 2016 at 1:59 PM, Lou Berger <[e-mail]<mailto:[e-mail]>
> <mailto:[e-mail]>> wrote:
>
>     Hi Alia,
>
>         Overlap looks likely.  I hope that when it does occur, you send
>     folks over here to contribute -- additional input / WG participation
>     would be great!  (For folks not familiar with DetNet, check out our
>     charter at https://datatracker.ietf.org/wg/detnet/charter/
>     <https://datatracker.ietf.org/wg/detnet/charter/>)
>
>     I expect we'll have some good data plane centric discussions at this
>     meeting, for those who may be interested (there's even still time to
>     submit related drafts!)
>
>     Cheers,
>
>     Lou
>
>     DetNet Co-chair
>
>     On 10/19/2016 11:46 AM, Alia Atlas wrote:
>     > Hi Patrick,
>     >
>     > The discussion intended on dlnex, a non-WG mailing list, is, I
>     think,
>     > broader - spanning the range of communications between upper layer
>     > applications and the network.  It is not clear yet what work might
>     > develop.  Be sure that I will be watching for any specific overlap
>     > with DetNet.
>     >
>     > I feel that it is useful to make it easy for groups to have a
>     list to
>     > brainstorm about their ideas before work is mature enough to
>     consider
>     > for standardization.
>     >
>     > Regards,
>     > Alia
>     >
>     > On Wed, Oct 19, 2016 at 8:17 AM, Patrick Wetterwald (pwetterw)
>     > <[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>
>     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>> wrote:
>     >
>     >     Can someone explain me why this mailing list is created
>     outside of
>     >     DetNet?
>     >
>     >     This sounds like a subset of Detnet no?
>     >
>     >     Thanks,
>     >
>     >     Patrick
>     >
>     >
>     >     On 19/10/2016, 07:56, "detnet on behalf of Pascal Thubert
>     >     (pthubert)" <[e-mail]<mailto:[e-mail]>
>     <mailto:[e-mail]>
>     >     <mailto:[e-mail]
>     <mailto:[e-mail]>> on behalf of [e-mail]<mailto:[e-mail]>
>     <mailto:[e-mail]>
>     >     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>> wrote:
>     >
>     >         From: IETF-Announce
>     [mailto:[e-mail]
>     <mailto:[e-mail]>
>     >     <mailto:[e-mail]
>     <mailto:[e-mail]>>] On Behalf Of IETF
>     Secretariat
>     >         Sent: mardi 18 octobre 2016 22:36
>     >         To: IETF Announcement List <[e-mail]<mailto:[e-mail]>
>     <mailto:[e-mail]>
>     >     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>>
>     >         Cc: [e-mail]<mailto:[e-mail]>
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>;
>     >     [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>;
>     [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     >     <mailto:[e-mail]
>     <mailto:[e-mail]>>
>     >         Subject: New Non-WG Mailing List: DLNEX -- Discussion of
>     >     reliable and deterministic latency attributes
>     >
>     >
>     >         A new IETF non-working group email list has been created.
>     >
>     >         List address:[e-mail]<mailto:address:[e-mail]>
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>
>     >         Archive:
>     >     https://mailarchive.ietf.org/arch/search/?email_list=dlnex
>
>     >     <https://mailarchive.ietf.org/arch/search/?email_list=dlnex>
>     >         To subscribe:
>     https://www.ietf.org/mailman/listinfo/dlnex
>     <https://www.ietf.org/mailman/listinfo/dlnex>
>     >     <https://www.ietf.org/mailman/listinfo/dlnex>
>     >
>     >         Purpose:
>     >         DLNEX is to discuss various latency characteristics that can
>     >     be exposed by network elements or segments and to explore if
>     there
>     >     are any latency related attributes that can be utilized by upper
>     >     layer. For example, could there be latency exposure that upper
>     >     layer can utilize to plan how to distribute their content to the
>     >     right edges to achieve optimal user experience? Or something
>     used
>     >     by Interactive AR controller to optimize their services? Is
>     there
>     >     any value gained by upper layer expressing that they would
>     rather
>     >     have fixed latency than losing packets?
>     >
>     >         The discussion is to answer questions like: are there any
>     >     effective interaction/coordination between upper layer and lower
>     >     layer to achieve more efficient optimization for latency
>     sensitive
>     >     services?
>     >
>     >         This discussion group is also a place to showcase the
>     state of
>     >     the arts in latency optimized switching architecture and
>     interface
>     >     designs, as the feasibility exercise for achieving reliable and
>     >     deterministic latency through a network element or a segment.
>     >     Those latency exposures are the foundation for (future) latency
>     >     optimized control plane.
>     >
>     >
>     >         For additional information, please contact the list
>     >     administrators.
>     >
>     >         _______________________________________________
>     >         detnet mailing list
>
>     >         [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>
>     >         https://www.ietf.org/mailman/listinfo/detnet
>     >     <https://www.ietf.org/mailman/listinfo/detnet>
>     >
>     >
>     >     _______________________________________________
>     >     detnet mailing list
>     >     [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]<mailto:[e-mail]%20%3Cmailto:[e-mail]>>>
>     >     https://www.ietf.org/mailman/listinfo/detnet
>
>     >     <https://www.ietf.org/mailman/listinfo/detnet>
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > detnet mailing list
>     > [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     > https://www.ietf.org/mailman/listinfo/detnet
>
>     _______________________________________________
>     detnet mailing list
>     [e-mail]<mailto:[e-mail]> <mailto:[e-mail]>
>     https://www.ietf.org/mailman/listinfo/detnet
>
>
>
>
>
> _______________________________________________
> detnet mailing list
> [e-mail]<mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/detnet





--

________________________________________________________________

Bob Briscoe                               http://bobbriscoe.net/