Re: [iccrg] [tsvwg] [CCWG] New Internet Draft: Congestion Signaling (CSIG)

[Bob Briscoe <[e-mail]>]

KK, Christian,

I want to focus for a moment on the problem of how a flow in progress 
knows that available capacity has increased (e.g. because other flow(s) 
have departed or bottleneck link capacity has increased). Current 
congestion control algorithms (CCAs) take so long to detect an absence 
of signals (loss, delay or ECN), that they use the same probing 
algorithm that they use in their congestion avoidance phase to fill any 
available capacity.

With Reno, Cubic, etc. the time between congestion episodes in steady 
state for a single flow is now hundreds of RTT. So, it takes even more 
than that before you can be sure there is a real absence of congestion 
signals.

Scalable CCAs do a lot better. The target average time between signals 
in stable conditions is half an RTT. You can sustain such a high 
signalling rate with ECN, but you couldn't do it with loss, which is why 
L4S is tightly associated with ECN. So, after just a few RTT of no 
signals you can start probing for more capacity more aggressively than 
you would want to in congestion avoidance phase.

This is the critical point - the CCAs of all flows that notice the 
absence of signals briefly enter a separate acceleration mode where 
convergence is a non-goal. Then, when they find a closed loop signal 
again, they return to their congestion avoidance mode to converge with 
each other from wherever they got to when they 'broke loose'.

Before Joakim Misund decided to move on from his doctoral research into 
getting up to speed fast, he did some nice experiments on this 
acceleration mode.
For instance, look at Fig 5 in this draft tech report he wrote:
https://bobbriscoe.net/projects/latency/04-delay_bound_faster_additive_increase_when_unterutilize_tr.pdf

In Fig 5 capacity alternates between 20 Mb/s and 200 Mb/s in a square 
wave. It compares his modified TCP Prague (the pink plot) with BBRv2 & 
unmodified Prague.  The legend calls the pink plot "Prague w/both" 
because it has an acceleration mode and an overload mode to rapidly 
decrease when there is both ECN and heavy delay.


Bob


On 16/09/2023 12:13, K. K. Ramakrishnan wrote:
> I agree. This was the original idea  for a "faster" ramp up initially, 
> Christian. It is (if available on the end-end path) a relatively 
> benign solution that doesn't stomp out existing flows.
> If ECN is not available on the path, it is easy enough to not ramp up.
>
> On Fri, Sep 15, 2023 at 9:19 PM Christian Huitema 
> <[e-mail]> wrote:
>
>     RFC 9049 is an interesting read. Reading it, I was nodding my head...
>     until it went to the ECN section. The big change we are seeing is
>     that
>     deploying ECN now is mostly safe -- QUIC, for example, can test for
>     end-to-end availability of ECN, and measurements show that it is
>     available on many paths. It may not be actively implemented by all
>     elements in the path, but at least nothing crashes, and we can see
>     deployments. It only took two decades to get there.
>
>     Which means that maybe we should start fully exploiting ECN for
>     what it
>     is worth. Probably not a complete solution to the "quick start"
>     problem,
>     but at least a partial solution, one that would complement packet
>     loss
>     detection and delay increase measurements. That seems more promising
>     than trying to deploy another solution across all Internet routers.
>
>     And maybe we can make creative use of ECN, even in cases of long
>     delay
>     links. Combine it with chirping, maybe as in "send a chirp at a quick
>     rate, and check whether that chirp elicited ECN feedback?"
>
>     -- Christian Huitema
>
>     On 9/15/2023 1:58 AM, Gorry Fairhurst wrote:
>     > I wonder if RFC 9049 - Path Aware Networking: Obstacles to
>     Deployment (A
>     > Bestiary of Roads Not Taken), section 6.3, might have some thoughts
>     > relevant to why some approaches in DC research diverge from the
>     > end-to-end approaches described in published RFCs.
>     >
>     > Best wishes,
>     > Gorry
>     >
>     >
>     > On 15/09/2023 08:37, Hesham ElBakoury wrote:
>     >> Speaking of Internet congestion control, have you looked at this
>     >> paper: Future Internet Congestion Control: The Diminishing
>     Feedback
>     >> Problem [https://arxiv.org/pdf/2206.06642]? The abstract of
>     this paper
>     >> says: /<we argue that congestion control mechanisms should move
>     away
>     >> from their strict “end-to-end”/
>     >> /adherence. This change would benefit from avoiding a “one size
>     fits
>     >> all circumstances” approach, and moving towards a more/
>     >> /selective set of mechanisms that will result in a better
>     performing
>     >> Internet./>
>     >>
>     >> Hesham
>     >>
>     >> On Thu, Sep 14, 2023, 3:23 AM Sebastian Moeller
>     <[e-mail]> wrote:
>     >>
>     >>     Hi Ruediger,
>     >>
>     >>
>     >>     > On Sep 14, 2023, at 09:25, <[e-mail]>
>     >>     <[e-mail]> wrote:
>     >>     >
>     >>     > Sebastian, Ingemar,
>     >>     >
>     >>     > the draft says "limited domain". The method suggested
>     resembles
>     >>     IPPM's IOAM (but the packet size remains fixed, certainly an
>     >>     advantage). Information like a port ID only makes sense, if the
>     >>     evaluation unit is aware of the domain-topology.
>     >>
>     >>             [SM] Not sure I understand this, if on an internet-wide
>     >>     path all nodes would simply include the minimum of their egress
>     >>     bitrate (that is only include this if it is lower than the
>     >>     existing value), then an end-2end protocol like TCP could
>     even as
>     >>     part of the initial 3 way handshake figure out a hard limit for
>     >>     maximum rasinsable cwin (by simply calculating the BDP).
>     SUre that
>     >>     would not stop slow-start fuully from overshooting, but it
>     should
>     >>     limit the maximum overshoot considerably....
>     >>
>     >>
>     >>     > That will be restricted to a single domain, I think.
>     >>
>     >>             [SM] As long as the forward and collection path
>     requires
>     >>     L2 elements, sure this will not work over the internet.
>     >>
>     >>
>     >>     > I'm also not sure, what is to be optimized on an operational
>     >>     carrier backbone.
>     >>
>     >>             [SM] I trust you on this, as that is out of my league.
>     >>
>     >>
>     >>     > Under expectable conditions, these are operated without
>     >>     congestion. Bottlenecks are likely at the access and, in a
>     rather
>     >>     limited number of cases, at peering. There you'd need to
>     know the
>     >>     RTT so that the available bandwidth calculation makes sense.
>     >>
>     >>             [SM] But the protocol that is supposed to consume the
>     >>     congestion information will know the RTT, so can take this into
>     >>     account, the network really only needs to give information
>     about
>     >>     its current state. That said, I can see network nodes that
>     might
>     >>     want to know a flow's RTT, but I do not think there is a way of
>     >>     supplying that this is not both easy and efficient and not
>     easily
>     >>     gamed. Passing on congestion informatin however is far less
>     >>     abusable (after all the node could just as well have
>     dropped the
>     >>     packet instead of bothering to fudge the congestion info).
>     >>
>     >>
>     >>     > Or you need to calculate several values for several RTTs and
>     >>     provide this information (RTTs and corresponding available
>     >>     bandwidths). Requires a fair amount of bytes, so then trust
>     may be
>     >>     an issue (I'm not sure whether a carrier would be willing
>     to add
>     >>     an extended Ethernet header to every encrypted IP packet
>     >>     indicating interest in getting CSIG information on
>     bottleneck links).
>     >>
>     >>             [SM] Yes encryption/privacy are reasons to stick to
>     L2/L3.
>     >>     I would guess in a limited domain deploying CSIG as a
>     pseudo VLAN
>     >>     header seems like a nice work-around, but this information
>     really
>     >>     should be inside IP header to go end to end. IMHO it also
>     should
>     >>     not be in an optional header, but should have been part of the
>     >>     mandatory header, but that ship has sailed long ago... It turns
>     >>     out that the mechanism intended here IPv6 extension headers was
>     >>     mis-deployed... (It should have been initiated with a really
>     >>     desirable use-case so it would actrually see usage immediately,
>     >>     protocol design is a bit like evolution, in both cases "use
>     it or
>     >>     loose it" applies to some degree.)
>     >>
>     >>     Regards
>     >>             Sebastian
>     >>
>     >>
>     >>     >
>     >>     > Regards, Ruediger
>     >>     >
>     >>     > -----Ursprüngliche Nachricht-----
>     >>     > Von: Sebastian Moeller <[e-mail]>
>     >>     > Gesendet: Donnerstag, 14. September 2023 08:49
>     >>     > An: Ingemar Johansson S <[e-mail]>
>     >>     > Cc: Geib, Rüdiger <[e-mail]>; [e-mail];
>     >> [e-mail]; [e-mail]; [e-mail]
>     >>     > Betreff: Re: [tsvwg] [iccrg] New Internet Draft: Congestion
>     >>     Signaling (CSIG)
>     >>     >
>     >>     > Hi Ingemar
>     >>     >
>     >>     >> On Sep 14, 2023, at 08:42, Ingemar Johansson S
>     >>     <[e-mail]> wrote:
>     >>     >>
>     >>     >> Hi Ruediger
>     >>     >>
>     >>     >> Even though CSIG is on ethernet, it appears to be e2e as the
>     >>     feedback is on L4. So I guess somehow the CSIG info needs
>     to jump
>     >>     from domain to domain somewhow, and that sounds to me like L3,
>     >>     albeit perhaps brief jumps ?
>     >>     >
>     >>     >       [SM] rfc3168 and L4S ECN signaling faces the same
>     hurdle,
>     >>     forward path uses a different layer than feed-back path.
>     And I do
>     >>     not think that it is conceptually all that cleaner to have
>     fewer
>     >>     layers between forward and reverse signaling... "clean"
>     would be
>     >>     to put both forward and reverse information into the same
>     layer,
>     >>     but that is not on the table as far as I can see.
>     >>     >       The other difference however, richness of signal, is a
>     >>     clear difference between 1 bit ECN and multibit CSIG (and
>     >>     alternatives). (Side-note, sure the ECN bitfield is two
>     bits but
>     >>     it only carries 2 congestion states when ECN is in use, which
>     >>     makes it IMHO a 1-bit information channel, if we had
>     followed the
>     >>     SCE proposal with its 3 congestion states we would be at
>     1.5 bits ;))
>     >>     >
>     >>     > Regards
>     >>     >       Sebastian
>     >>     >
>     >>     >
>     >>     >
>     >>     >>
>     >>     >> /Ingemar
>     >>     >>
>     >>     >>> -----Original Message-----
>     >>     >>> From: [e-mail] <[e-mail]>
>     >>     >>> Sent: Wednesday, 13 September 2023 14:20
>     >>     >>> To: [e-mail]; Ingemar Johansson S
>     >>     >>> <[e-mail]>
>     >>     >>> Cc: [e-mail]; [e-mail];
>     [e-mail];
>     >>     >>> [e-mail]; [e-mail]; [e-mail];
>     >>     >>> [e-mail]; [e-mail]; [e-mail];
>     >>     >>> [e-mail]; [e-mail]; [e-mail]
>     >>     >>> Subject: AW: [tsvwg] [iccrg] New Internet Draft: Congestion
>     >>     Signaling
>     >>     >>> (CSIG)
>     >>     >>>
>     >>     >>> Hi Ingemar, hi Sebastian,
>     >>     >>>
>     >>     >>> Skimming over the draft only, isn't CSIG about
>     Ethernet-domains,
>     >>     >>> while L4S is E2E?
>     >>     >>>
>     >>     >>> Regards,
>     >>     >>>
>     >>     >>> Ruediger
>     >>     >>>
>     >>     >>> -----Ursprüngliche Nachricht-----
>     >>     >>> Von: tsvwg <[e-mail]> Im Auftrag von
>     Sebastian
>     >>     Moeller
>     >>     >>> Gesendet: Mittwoch, 13. September 2023 13:42
>     >>     >>> An: Ingemar Johansson S
>     >>     >>> <ingemar.s.johansson=[e-mail]>
>     >>     >>> Cc: Vidhi Goel <vidhi_goel=[e-mail]>;
>     >>     >>> Shihang(Vincent)
>     <shihang9=[e-mail]>; tsvwg
>     >>     >>> <[e-mail]>; Jai Kumar <[e-mail]>; IETF
>     >>     IPPM WG
>     >>     >>> <[e-mail]>; Tom Herbert
>     >>     <tom=[e-mail]>;
>     >>     >>> [e-mail]; Abhiram Ravi
>     >>     <abhiramr=[e-mail]>;
>     >>     >>> Nandita Dukkipati <[e-mail]>; [e-mail];
>     >>     Huangyihong
>     >>     >>> (Rachel) <rachel.huang=[e-mail]>;
>     Naoshad
>     >>     Mehta
>     >>     >>> <[e-mail]>
>     >>     >>> Betreff: Re: [tsvwg] [iccrg] New Internet Draft: Congestion
>     >>     Signaling
>     >>     >>> (CSIG)
>     >>     >>>
>     >>     >>> Hi Ingemar,
>     >>     >>>
>     >>     >>>
>     >>     >>>> On Sep 13, 2023, at 12:30, Ingemar Johansson S
>     >>     >>> <ingemar.s.johansson=[e-mail]> wrote:
>     >>     >>>>
>     >>     >>>> Hi
>     >>     >>>>
>     >>     >>>> I agree with Vihdi
>     >>     >>>>
>     >>     >>>> L4S is recently standardised
>     >>     >>>
>     >>     >>>     [SM] In experimental track, the goal is currently
>     to test
>     >>     whether it
>     >>     >>> can/should be deployed at scale....
>     >>     >>>
>     >>     >>>> and it is definitely gaining traction also in 3GPP. We
>     have
>     >>     an echo
>     >>     >>> system that is looking forward to having L4S widely
>     deployed.
>     >>     >>>> Still, the congestion control aspects are not fully
>     explored
>     >>     yet.
>     >>     >>>> One
>     >>     >>> interesting topic is if L4S allows to more safely
>     deviate from
>     >>     >>> additive increase to make congestion control algorithms
>     more
>     >>     quickly
>     >>     >>> converge to higher link capacity. There are a number of
>     study
>     >>     topics
>     >>     >>> around L4S congestion control that are listed in e.g
>     the TCP
>     >>     Prague draft.
>     >>     >>>>
>     >>     >>>> I cannot dictate what others should do with their time and
>     >>     money but
>     >>     >>> personally I'd prefer that the IETF explores L4S and its
>     >>     >>> possibilities and downsides before jumping on the next
>     idea.
>     >>     >>>
>     >>     >>>     [SM] L4S can be described as taking the ideas
>     behind DCTCP
>     >>     and
>     >>     >>> making them fit for use over the internet*. Yet the
>     signaling
>     >>     >>> discussed here is to be used in e.g. data center contexts
>     >>     where DCTCP
>     >>     >>> is already used and found lacking compared to newer methods
>     >>     operating
>     >>     >>> on richer congestion information (HPCC, Swift,
>     Poseidon, ...).
>     >>     >>>     Given that L4S essentially uses a multi-packet
>     signal(**)
>     >>     to report
>     >>     >>> the "queue filling state" that is then stochastically
>     distributed
>     >>     >>> over all concurrent flows, it seems obvious to me that
>     >>     reconstructing
>     >>     >>> a reliable estimate of on-path queueing will take some
>     time and
>     >>     >>> averaging for each individual flow, I would guess that
>     in some
>     >>     >>> environments this delay simply is too costly.
>     >>     >>>     So L4S and CSIG seem complementary and in no way
>     mutually
>     >>     exclusive.
>     >>     >>>
>     >>     >>>
>     >>     >>> Regards
>     >>     >>>     Sebastian
>     >>     >>>
>     >>     >>>
>     >>     >>>
>     >>     >>> *) I will not further discuss whether that is achieved
>     or not
>     >>     as it
>     >>     >>> seems irrelevant here.
>     >>     >>> **) In essence transmission of congestion state via a 1-bit
>     >>     serial
>     >>     >>> channel, clocked at the (variable***) packet rate at the
>     >>     bottleneck.
>     >>     >>> ***) as packets are not of uniform size
>     >>     >>>
>     >>     >>>>
>     >>     >>>> CSIG sounds to me like something that belongs more in
>     ICCRG or ?
>     >>     >>>>
>     >>     >>>> /Ingemar
>     >>     >>>>
>     >>     >>>> From: tsvwg <[e-mail]> On Behalf Of
>     Vidhi Goel
>     >>     >>>> Sent: Wednesday, 13 September 2023 00:59
>     >>     >>>> To: Shihang(Vincent)
>     <shihang9=[e-mail]>
>     >>     >>>> Cc: Huangyihong (Rachel)
>     >>     <rachel.huang=[e-mail]>;
>     >>     >>> Tom Herbert <tom=[e-mail]>;
>     Abhiram Ravi
>     >>     >>> <abhiramr=[e-mail]>; IETF IPPM WG
>     >>     <[e-mail]>;
>     >>     >>> tsvwg <[e-mail]>; [e-mail]; [e-mail];
>     Nandita
>     >>     >>> Dukkipati <[e-mail]>; Naoshad Mehta
>     >>     <[e-mail]>;
>     >>     >>> Jai Kumar <[e-mail]>
>     >>     >>>> Subject: Re: [tsvwg] [iccrg] New Internet Draft:
>     Congestion
>     >>     >>>> Signaling
>     >>     >>> (CSIG)
>     >>     >>>>
>     >>     >>>> Not sure why we are coming up with so many new techniques
>     >>     when ECN
>     >>     >>> just works fine.
>     >>     >>>> ECN is a 2 bit field (not 1 bit) and seems to be
>     sufficient to
>     >>     >>> indicate extent of congestion by marking it per packet.
>     Adding
>     >>     more
>     >>     >>> complexity to any layer whether it is L2 or L3 doesn’t work
>     >>     well in
>     >>     >>> deployments. Our goal should be to simplify things and only
>     >>     add new
>     >>     >>> headers if absolutely necessary.
>     >>     >>>>
>     >>     >>>> Vidhi
>     >>     >>>>
>     >>     >>>>
>     >>     >>>> On Sep 12, 2023, at 3:12 AM, Shihang(Vincent)
>     >>     >>> <shihang9=[e-mail]> wrote:
>     >>     >>>>
>     >>     >>>> Hi,
>     >>     >>>> I agree L2 may not be the best choice to carry the
>     congestion
>     >>     >>> signaling end-to-end and more bits are needed. We have
>     >>     submitted a
>     >>     >>> draft to carry the multi-bits congestion signaling in
>     L3. We
>     >>     call it
>     >>     >>> Advanced ECN. See
>     >> https://datatracker.ietf.org/doc/draft-shi-ccwg-advanced-ecn/.
>     >>     >>>>
>     >>     >>>> Thanks,
>     >>     >>>> Hang
>     >>     >>>>
>     >>     >>>> From: CCWG <[e-mail]> On Behalf Of
>     Huangyihong
>     >>     (Rachel)
>     >>     >>>> Sent: Tuesday, September 12, 2023 5:41 PM
>     >>     >>>> To: Tom Herbert <tom=[e-mail]>;
>     >>     Abhiram Ravi
>     >>     >>> <abhiramr=[e-mail]>
>     >>     >>>> Cc: IETF IPPM WG <[e-mail]>; tsvwg <[e-mail]>;
>     >>     >>> [e-mail]; [e-mail]; Nandita Dukkipati
>     >>     >>> <[e-mail]>; Naoshad Mehta
>     <[e-mail]>; Jai
>     >>     Kumar
>     >>     >>> <[e-mail]>
>     >>     >>>> Subject: Re: [CCWG] [iccrg] [tsvwg] New Internet Draft:
>     >>     Congestion
>     >>     >>> Signaling (CSIG)
>     >>     >>>>
>     >>     >>>> Hi,
>     >>     >>>>
>     >>     >>>> I also have the same feeling. Implementing in L2 may be
>     >>     difficult to
>     >>     >>> be used in e2e transport. Of course it can work well in
>     limited
>     >>     >>> domain, like DC or HPC clusters. However, I also look
>     for some
>     >>     >>> solutions that could be able to go through internet. We
>     have
>     >>     >>> submitted a draft to describe the transport challenges. See
>     >>     >>>
>     >> https://datatracker.ietf.org/doc/html/draft-huang-tsvwg-transport-
>     >>     >>> challenges.
>     >>     >>>>
>     >>     >>>> I share the same opinion that the congestion signal is
>     useful
>     >> and
>     >>     >>> current 1-bit ECN solution is not fully sufficient. But
>     I also
>     >>     feel
>     >>     >>> like the more straight way is to extend L3, or l4, like
>     update
>     >>     IOAM,
>     >>     >>> to carry the information. For L2 solution, it should be
>     developed
>     >>     >>> together with IEEE 802.1.
>     >>     >>>>
>     >>     >>>> BR，
>     >>     >>>> Rachel
>     >>     >>>>
>     >>     >>>> 发件人: iccrg <[e-mail]> 代表 Tom Herbert
>     >>     >>>> 发送时间: 2023年9月10日 0:10
>     >>     >>>> 收件人: Abhiram Ravi
>     <abhiramr=[e-mail]>
>     >>     >>>> 抄送: IETF IPPM WG <[e-mail]>; tsvwg
>     <[e-mail]>;
>     >>     >>> [e-mail]; [e-mail]; Nandita Dukkipati
>     >>     >>> <[e-mail]>; Naoshad Mehta
>     <[e-mail]>; Jai
>     >>     Kumar
>     >>     >>> <[e-mail]>
>     >>     >>>> 主题: Re: [iccrg] [tsvwg] New Internet Draft: Congestion
>     >>     Signaling
>     >>     >>> (CSIG)
>     >>     >>>>
>     >>     >>>> Hi, thanks for draft!
>     >>     >>>>
>     >>     >>>> The first thing that stands out to me is the carrier
>     of the new
>     >>     >>>> packet
>     >>     >>> headers. In the forward path it would be in L2 and in
>     >>     reflection it
>     >>     >>> would be L4. As the draft describes, this would entail
>     having to
>     >>     >>> support the protocol in multiple L2 and multiple L4
>     protocols--
>     >>     >>> that's going to be a pretty big lift! Also, L2 is not
>     really an
>     >>     >>> end-to-end protocol (would legacy switches in the path also
>     >>     forward the header)l?).
>     >>     >>>>
>     >>     >>>> The signaling being described in the draft is network
>     layer
>     >>     >>> information, and hence IMO should be conveyed in
>     network layer
>     >>     headers.
>     >>     >>> That's is L3 which conveniently is the average of L2+L4 :-)
>     >>     >>>>
>     >>     >>>> IMO, the proper carrier of the signal data is Hop-by-Hop
>     >>     Options.
>     >>     >>>> This
>     >>     >>> is end-to-end and allows modification of data
>     in-flight. The
>     >>     typical
>     >>     >>> concern with Hop-by-Hop Options is high drop rates on the
>     >>     Internet,
>     >>     >>> however in this case the protocol is explicitly
>     confined to a
>     >>     limited
>     >>     >>> domain so I don't see that as a blocking issue for this
>     use case.
>     >>     >>>>
>     >>     >>>> The information being carried seems very similar to
>     that of IOAM
>     >>     >>>> (IOAM
>     >>     >>> uses Hop-by-Hop Options and supports reflection). I
>     suppose the
>     >>     >>> differences are that this protocol is meant to be
>     consumed by the
>     >>     >>> transport Layer and the data is a condensed summary of path
>     >>     >>> characteristics. IOAM seems pretty extensible, so maybe it
>     >>     could be
>     >>     >>> adapted to carry the signals of this draft?
>     >>     >>>>
>     >>     >>>> A related proposal might be FAST draft-herbert-fast. Where
>     >>     the CSIG
>     >>     >>>> is
>     >>     >>> network to host signaling, FAST is host to network
>     signaling
>     >>     for the
>     >>     >>> purposes of requesting network services. These might be
>     >>     complementary
>     >>     >>> and options for both may be in the same packet. FAST
>     also uses
>     >>     >>> reflection, so we might be able to leverage some common
>     >>     >>> implementation at a destination.
>     >>     >>>>
>     >>     >>>> Tom
>     >>     >>>>
>     >>     >>>> On Fri, Sep 8, 2023, 7:43 PM Abhiram Ravi
>     >>     >>> <abhiramr=[e-mail]> wrote:
>     >>     >>>> Hi IPPM folks,
>     >>     >>>>
>     >>     >>>> I am pleased to announce the publication of a new
>     internet
>     >> draft,
>     >>     >>> Congestion Signaling (CSIG):
>     >> https://datatracker.ietf.org/doc/draft-
>     >>     >>> ravi-ippm-csig/
>     >>     >>>>
>     >>     >>>> CSIG is a new end-to-end packet header mechanism for
>     in-band
>     >>     >>>> signaling
>     >>     >>> that is simple, efficient, deployable, and grounded in
>     >>     concrete use
>     >>     >>> cases of congestion control, traffic management, and
>     network
>     >>     >>> debuggability. We believe that CSIG is an important new
>     >>     protocol that
>     >>     >>> builds on top of existing in-band network telemetry
>     protocols.
>     >>     >>>>
>     >>     >>>> We encourage you to read the CSIG draft and provide your
>     >>     feedback
>     >>     >>>> and
>     >>     >>> comments. We have also cc'd the TSVWG, CCWG, and ICCRG
>     mailing
>     >>     lists,
>     >>     >>> as we believe that this work may be of interest to their
>     >>     members as well.
>     >>     >>>>
>     >>     >>>> Thank you for your time and consideration.
>     >>     >>>>
>     >>     >>>> Sincerely,
>     >>     >>>> Abhiram Ravi
>     >>     >>>> On behalf of the CSIG authors
>     >>     >>
>     >>     >
>     >>     > _______________________________________________
>     >>     > iccrg mailing list
>     >>     > [e-mail]
>     >>     > https://www.irtf.org/mailman/listinfo/iccrg
>     >>
>     >>     --     CCWG mailing list
>     >> [e-mail]
>     >> https://www.ietf.org/mailman/listinfo/ccwg
>     >>
>     >
>     >
>     > _______________________________________________
>     > iccrg mailing list
>     > [e-mail]
>     > https://www.irtf.org/mailman/listinfo/iccrg
>
>     _______________________________________________
>     iccrg mailing list
>     [e-mail]
>     https://www.irtf.org/mailman/listinfo/iccrg
>
>
>
> -- 
> K. K. Ramakrishnan
>
> _______________________________________________
> iccrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/